## 5 Results
### Defenses Against Adversarial Examples
#### 5.3.4 Detecting Adversarial Examples  
Much research has considered the best way to detect adversarial examples. If adversarial examples can be detected they could be more easily deflected, and perhaps even the original input could be salvaged and correctly classified. [PERSON] et al. [124] proposed a statistical test to detect adversarial examples before they are input into machine learning models. They observed that adversarial examples are unrepresentative of the distribution, and lie in unexpected regions of a model's output surface. Their proposed outlier detection system relies on the statistical separation of adversarial examples. They subsequently evaluated their model against adaptive strategies and strong black-box strategies. [PERSON] et al. [125] proposed a binary classifier \"detector subnetwork\" aiming to distinguish between genuine data and adversarial examples. The detection of adversarial examples does not unequivocally lead to correct classification; however, the effect of adversarial examples could perhaps be mitigated through fallback solutions, for example, by requesting human intervention. After successfully detecting adversarial examples in their experiments, they later bypassed their defenses by generating adversarial examples that fool both detector and classifier. They further proposed a training procedure called \"dynamic adversary training\" as a countermeasure to their attack against the detector. [PERSON] et al. [126] also detected adversarial examples by considering which artifacts of adversarial examples could help detection. They considered two complementary features used to detect adversarial examples: density estimates and Bayesian uncertainty estimates. They evaluated these features on CNNs trained on MNIST and CIFAR-10 datasets. They effectively detected adversarial examples with an ROC-AUC of 92.6%. They further suggested that their method could be used in RNNs. This suggestion is bolstered by [PERSON] and [PERSON]'s [144] assertion that Bayesian approximation using dropout can be applied to RNN networks. [PERSON] et al. [127] proposed a framework, \"MagNET\", to detect adversarial examples. This framework precedes the classifier it defends. The framework has two components: (1) A detector finds and discards any out-of-distribution examples (those significantly far from the manifold boundary); (2) A reformer that aims to find close approximations to inputs before forwarding the approximations to the classifier. Their system generalizes well because it learns to detect adversarial examples without knowledge of how they were generated. They proposed a defense against gray-box attacks where the adversary has knowledge of the deployed defenses. The proposed defense trains a number of auto-encoders (or _reformers_). At test-time a single auto-encoder is selected at random. [PERSON] et al. [128] proposed \"feature squeezing\" as a strategy to detect adversarial examples by squeezing out unnecessary features in the input. Through comparing predictions of the original and feature squeezed inputs, adversarial examples are identified if the difference between the two predictions meets a threshold. Two feature-squeezing methods are used: (1) Reducing the colour bit-depth of the image; (2) Spatial smoothing. An adversary may adapt and circumvent this defense; however, the defense may frustrate the adversary because it changes the problem the adversary must overcome. [PERSON] et al. [136] considered the feature squeezing defense designed for CNNs and proposed \"sequence squeezing\", which is adapted for RNNs. Adversarial examples are similarly detected by running the classifier twice: once on the original sequence, and once for the sequence-squeezed input. An input is identified as adversarial if the difference in the confidence scores meets a threshold value. [PERSON] et al. [141] proposed an image classifier based on a variational auto-encoder. They trained two models each on half the dataset: a target model and a surrogate model.