Foolbox: A Python toolbox to benchmark the robustness of machine learning models

Even todays most advanced machine learning models are easily fooled by almost imperceptible perturbations of their inputs. Foolbox is a new Python package to generate such adversarial perturbations and to quantify and compare the robustness of machine learning models. It is build around the idea that the most comparable robustness measure is the minimum perturbation needed to craft an adversarial example. To this end, Foolbox provides reference implementations of most published adversarial attack methods alongside some new ones, all of which perform internal hyperparameter tuning to find the minimum adversarial perturbation. Additionally, Foolbox interfaces with most popular deep learning frameworks such as PyTorch, Keras, TensorFlow, Theano and MXNet and allows different adversarial criteria such as targeted misclassification and top-k misclassification as well as different distance measures. The code is licensed under the MIT license and is openly available at https://github.com/bethgelab/foolbox. The most up-to-date documentation can be found at http://foolbox.readthedocs. io. In 2013, Szegedy et al. demonstrated that minimal perturbations, often almost imperceptible to humans, can have devastating effects on machine predictions. These so-called adversarial perturbations thus demonstrate a striking difference between human and machine perception. As a result, adversarial perturbations have been subject to many Equal contribution Centre for Integrative Neuroscience, University of Tübingen, Germany Bernstein Center for Computational Neuroscience, Tübingen, Germany International Max Planck Research School for Intelligent Systems, Tübingen, Germany Max Planck Institute for Biological Cybernetics, Tübingen, Germany Institute for Theoretical Physics, University of Tübingen, Germany. Correspondence to: Jonas Rauber <jonas.rauber@bethgelab.org>. Reliable Machine Learning in the Wild Workshop, 34 th International Conference on Machine Learning, Sydney, Australia, 2017. studies concerning the generation of such perturbations and strategies to protect machine learning models such as deep neural networks against them. A practical definition of the robustness R of a model, first used by Szegedy et al. (2013), is the average size of the minimum adversarial perturbation ρ(x) across many samples x, R = 〈ρ(x)〉 x where (1) ρ(x) = min δ d(x,x+ δ) s.t. x+ δ is adversarial (2) and d(·) is some distance measure. Unfortunately, finding the global minimum adversarial perturbation is close to impossible in any practical setting, and we thus employ heuristic attacks to find a suitable approximation. Such heuristics, however, can fail, in which case we could easily be mislead to believe that a model is robust (Brendel & Bethge, 2017). Our best strategy is thus to employ as many attacks as possible, and to use the minimal perturbation found across all attacks as an approximation to the true global minimum. At the moment, however, such a strategy is severely obstructed by two problems: first, the code for most known attack methods is either not available at all, or only available for one particular deep learning framework. Second, implementations of the same attack often differ in many details and are thus not directly comparable. Foolbox improves upon the existing Python package cleverhans by Papernot et al. (2016b) in three important aspects: 1. It interfaces with most popular machine learning frameworks such as PyTorch, Keras, TensorFlow, Theano, Lasagne and MXNet and provides a straight forward way to add support for other frameworks, 2. it provides reference implementations for more than 15 adversarial attacks with a simple and consistent API, and 3. it supports many different criteria for adversarial examples, including custom ones. This technical report is structured as follows: In section 1 we provide an overview over Foolbox and demonstrate Foolbox: A Python toolbox to benchmark the robustness of machine learning models how to benchmark a model and report the result. In section 2 we describe the adversarial attack methods that are implemented in Foolbox and explain the internal hyperparameter tuning.