  Deep neural networks are vulnerable to adversarial examples that mislead the
models with imperceptible perturbations. Though adversarial attacks have
achieved incredible success rates in the white-box setting, most existing
adversaries often exhibit weak transferability in the black-box setting,
especially under the scenario of attacking models with defense mechanisms. In
this work, we propose a new method called variance tuning to enhance the class
of iterative gradient based attack methods and improve their attack
transferability. Specifically, at each iteration for the gradient calculation,
instead of directly using the current gradient for the momentum accumulation,
we further consider the gradient variance of the previous iteration to tune the
current gradient so as to stabilize the update direction and escape from poor
local optima. Empirical results on the standard ImageNet dataset demonstrate
that our method could significantly improve the transferability of
gradient-based adversarial attacks. Besides, our method could be used to attack
ensemble models or be integrated with various input transformations.
Incorporating variance tuning with input transformations on iterative
gradient-based attacks in the multi-model setting, the integrated method could
achieve an average success rate of 90.1% against nine advanced defense methods,
improving the current best attack performance significantly by 85.1% . Code is
available at https://github.com/JHL-HUST/VT.
