Deep learning has shown impressive performance on hard perceptual problems. However, researchers found deep learning systems to be vulnerable to small, specially crafted perturbations that are imperceptible to humans. Such perturbations cause deep learning systems to mis-classify adversarial examples, with potentially disastrous consequences where safety or security is crucial. Prior defenses against adversarial examples either targeted specific attacks or were shown to be ineffective. We propose MagNet, a framework for defending neural network classifiers against adversarial examples. MagNet neither modifies the protected classifier nor requires knowledge of the process for generating adversarial examples. MagNet includes one or more separate detector networks and a reformer network. The detector networks learn to differentiate between normal and adversarial examples by approximating the manifold of normal examples. Since they assume no specific process for generating adversarial examples, they generalize well. The reformer network moves adversarial examples towards the manifold of normal examples, which is effective for correctly classifying adversarial examples with small perturbation. We discuss the intrinsic difficulties in defending against whitebox attack and propose a mechanism to defend against graybox attack. Inspired by the use of randomness in cryptography, we use diversity to strengthen MagNet. We show empirically that MagNet is effective against the most advanced state-of-the-art attacks in blackbox and graybox scenarios without sacrificing false positive rate on normal examples. • Security and privacy → Domain-specific security and privacy architectures; • Computing methodologies → Neural networks; KEYWORDS adversarial example, neural network, autoencoder Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Figure 1: An illustration of the reformer's effect on adversarial perturbations. The second row displays adversarial examples generated from the original normal examples in the first row by Carlini's L ∞ attack. The third row shows their perturbations against the original examples, and these perturbations lack prominent patterns. The fourth row displays the adversarial examples after being reformed by MagNet. The fifth row displays the remaining perturbations in the reformed examples against their original examples in the first row, and these perturbations have the shapes of their original examples.
Magnet REF trains a classifier on manifolds of normal examples to detect adversarial perturbations without prior knowledge of the attack.
MagNet: A Two-Pronged Defense against Adversarial Examples
CCS '17
Computer Science