# ShuffleDetect: Detecting Adversarial Images against Convolutional Neural Networks
## 1 Introduction  
Convolutional neural networks (CNNs) trained on large sets of examples are dominant tools for object recognition [1]. Although CNNs are capable of accurately classifying new images into object categories, they can nevertheless be deceived by adversarial attacks [2], whose strategies generally consist of altering inputs with perturbations that lead to classification errors. These attacks can be classified in terms of the amount of information that the attackers have at their disposal. Gradient-based attacks (e.g., [3; 4; 5; 6]) require information about the CNN's architecture and weights. Transfer-based attacks (e.g., [7; 8; 9]) require less insider knowledge about the CNN but query the CNN for a set of inputs, and the collected information is used to create a substitute model, similar to the targeted CNN. This substitute model is attacked by gradient-based methods, leading to adversarial images that also fool the target CNN. Score-based attacks (see [10]) are even less demanding. They do not have access to the training data, model architecture, or CNN parameters. They only make use of the CNN's predicted output probabilities for all or a subset of object classes. Ideally, security issues posed by adversarial attacks are prevented by methods that detect malicious input images, potentially exclude them from further processing by the CNN, and alert the user. Such detectors may be tailor-made for a specific type of attack or applied efficiently to a large variety of attacks. Their performances are encompassed by a series of indicators that assess how far their outputs can be trusted, and the memory overhead, time, or complexity required to finish their tasks. These detectors can be classified as supervised and unsupervised. On the one hand, supervised techniques have knowledge of adversarial images, and may attempt to reinforce CNNs by adding adversarial images to the training set [4]. These techniques are particularly effective when attacks are known in advance. On the other hand, unsupervised techniques [11; 12; 13; 14] operate without prior access to adversarial images. Instead, they apply transformations to the input image and analyze the consistency of predictions between the input image and its transformed versions. These techniques operate on the premise that CNNs maintain consistent predictions for clean images. This paper introduces ShuffleDetect as a new unsupervised method for the detection of adversarial images; it is simple to implement and works efficiently against adversarial images created by a series of 8 different attacks applied to 10 different ImageNet-trained CNNs. To summarize, given image \(\mathcal{I}\), ShuffleDetect assesses whether \(\mathcal{I}\) (resized to a square \(N\times N\) image, if necessary, to fit the CNN's input size) is adversarial or not for a given CNN \(\mathcal{C}\). Firstly, the algorithm extracts the dominating category \(\mathrm{Dom}_{\mathcal{C}}(\mathcal{I})\) in which \(\mathcal{C}\) classifies \(\mathcal{I}\). Secondly, the algorithm essentially \"splits\" \(\mathcal{I}\) into non-overlapping patches of an equal size \(s\times s\). Thirdly, for each permutation \(\sigma\) of a set of \(t\) permutations of these patches, the algorithm creates a shuffled image \(sh(\mathcal{I},\sigma)\), and requires from \(\mathcal{C}\) the dominating category \(\mathrm{Dom}_{\mathcal{C}}(sh(\mathcal{I},\sigma))\) in which \(\mathcal{C}\) classifies the shuffled image. Lastly, the algorithm compares the outcome with \(\mathrm{Dom}_{\mathcal{C}}(\mathcal{I})\). The detector classifies an input image \(\mathcal{I}\) as \"adversarial\" if the proportion of permutations \(\sigma\) among \(t\) permutations is such that the dominant categories of \(\mathcal{I}\) and \(sh_{\sigma}(I,s)\) differ by more than a certain threshold value \(R_{th}\). The remainder of this paper is organized as follows.