  Deep neural networks have been proved that they are vulnerable to adversarial
examples, which are generated by adding human-imperceptible perturbations to
images. To defend these adversarial examples, various detection based methods
have been proposed. However, most of them perform poorly on detecting
adversarial examples with extremely slight perturbations. By exploring these
adversarial examples, we find that there exists compliance between
perturbations and prediction confidence, which guides us to detect
few-perturbation attacks from the aspect of prediction confidence. To detect
both few-perturbation attacks and large-perturbation attacks, we propose a
method beyond image space by a two-stream architecture, in which the image
stream focuses on the pixel artifacts and the gradient stream copes with the
confidence artifacts. The experimental results show that the proposed method
outperforms the existing methods under oblivious attacks and is verified
effective to defend omniscient attacks as well.
