Defending against black-box adversarial attacks with gradient-free trained sign activation neural networks

While machine learning models today can achieve high accuracies on classification tasks , they can be deceived by minor imperceptible distortions to the data . These are known as adversarial attacks and can be lethal in the black-box setting which does not require knowledge of the target model type or its parameters . Binary neural networks that have sign activation and are trained with gradient descent have been shown to be harder to attack than conventional sigmoid activation networks but their improvements are marginal . We instead train sign activation networks with a novel gradient-free stochastic coordinate descent algorithm and propose an ensemble of such networks as a defense model . We evaluate the robustness of our model ( a hard problem in itself ) on image , text , and medical ECG data and find it to be more robust than ensembles of binary , full precision , and convolutional neural networks , and than random forests while attaining comparable clean test accuracy . In order to explain our model ’ s robustness we show that an adversary targeting a single network in our ensemble fails to attack ( and thus non-transferable to ) other networks in the ensemble . Thus a datapoint requires a large distortion to fool the majority of networks in our ensemble and is likely to be detected in advance . This property of non-transferability arises naturally from the non-convexity of sign activation networks and randomization in our gradient-free training algorithm without any adversarial defense effort . 1 INTRODUCTION . State of the art machine learning algorithms can achieve high accuracies in classification tasks but misclassify minor perturbations in the data known as as adversarial attacks Goodfellow et al . ( 2015 ) ; Papernot et al . ( 2016b ) ; Kurakin et al . ( 2016 ) ; Carlini & Wagner ( 2017 ) ; Brendel et al . ( 2018 ) . Adversarial examples have been shown to transfer across models which makes it possible to perform transfer-based ( substitute model ) black box attacks Papernot et al . ( 2016a ) . To counter adversarial attacks many defense methods been proposed with adversarial training being the most popular Szegedy et al . ( 2014 ) ; Tramèr et al . ( 2018 ) . However this tends to lower accuracy on clean test data that has no perturbations Raghunathan et al . ( 2019 ) ; Zhang et al . ( 2019 ) and can still be attacked with better transfer based methods Wu et al . ( 2020 ) ; Xie et al . ( 2019a ) ; Dong et al . ( 2019 ) . Many previously proposed defenses have also been shown to be vulnerable Carlini & Wagner ( 2017 ) ; Athalye et al . ( 2018 ) ; Ghiasi et al . ( 2020 ) thus leaving adversarial robustness an open problem in machine learning . A more lethal and practical attack than substitute model training is a boundary based one that requires only the prediction of the model Brendel et al . ( 2018 ) . These attacks are aimed at finding the minimum distortion to an image such that it will fool a classifier . This is in fact an NP-hard problem for ReLu activated neural networks Katz et al . ( 2017 ) ; Sinha et al . ( 2018 ) and tree ensemble classifiers Kantchelian et al . ( 2016 ) . Even approximating the minimum distortion for ReLu activated neural networks is NP-hard Weng et al . ( 2018 ) . Boundary based black box attacks such as HopSkipJump Chen et al. , Boundary Attack Brendel et al . ( 2018 ) and RayS Chen & Gu ( 2020 ) give an upper bound on the minimum adversarial distortion . Binary neural networks that have sign activation and binary weights were originally proposed as lightweight models . These are trained with gradient descent by approximating the sign activation . Recent work has shown that they tend to be more adversarially robust than full precision networks but the improvements are marginal ( see Tables 4 and 5 in Galloway et al . ( 2018 ) and Table 8 in Panda et al . ( 2019 ) ) . In this paper we propose a gradient free stochastic coordinate descent algorithm for training sign activation networks with and without binary weights similar to recent work Xue et al . ( 2020a ; b ) ; Xie et al . ( 2019b ) . While our original intention was to study the accuracy of a sign activation network trained directly without any approximation we make an interesting finding on the adversarial robustness of our model . We find that ensembling our model gives a high minimum distortion ( as measured by HopSkipJump ) compared to full precision , binary , and convolutional neural networks . We explain this phenomena by measuring the transferability between networks in an ensemble . In summary we make the following observations in our paper : • Our single hidden layer sign activation network has higher minimum distortion than ensembles of full precision and binary neural networks , than random forests that have the advantage of bootstrapping and random feature selection , and than ensembles of convolutional networks that have the advantage of convolutions and several layers . • Our model ’ s robustness stems from non-transferability of adversarial examples between networks in our ensemble and its robustness increases as we add more networks to the ensemble . • Substitute model black box attacks require a much greater distortion to bring our model to zero adversarial accuracy compared to ensembles of full precision and binary networks . • Text classification black box attacks are less effective on our model than on convolutional networks , random forests , and ensembles of full precision and binary networks . • In a medical diagnosis setting , attacks on ECG data on our model have a higher distortions and are visually distinguishable compared to attacks on ensembles of full precision and convolutional networks , and on random forests . 2 METHODS . 2.1 GRADIENT-FREE STOCHASTIC COORDINATE DECENT . Suppose we are given binary class data xi ∈ Rd and yi ∈ { −1 , +1 } for i = 0 ... n − 1 . Consider the objective function of a single hidden layer neural network with sign activation and 01 loss given below . We employ a stochastic coordinate descent algorithm shown in Algorithm 1 ( similar to recent work Xue et al . ( 2020a ; b ) ; Xie et al . ( 2019b ) ) to minimize this objective . 1 2n argmin W , W0 , w , w0 ∑ i ( 1− sign ( yi ( wT ( sign ( WTxi +W0 ) ) + w0 ) ) ) ( 1 ) We can train sign activation networks with and without binary weights using our SCD training procedure above . In the case of binary weights we don ’ t need a learning rate . We apply GPU parallelism to simultaneously update features and other heuristics to speed up runtimes ( with additional details given in the Supplementary Material ) . 2.2 IMPLEMENTATION , TEST ACCURACY , AND RUNTIME . We implement our training procedure in Python , numpy , and Pytorch Paszke et al . ( 2019 ) and make our code freely available from https : //github.com/zero-one-loss/scd_github . We train three types of sign activation networks with our algorithm : ( 1 ) SCD01 : 01-loss in the final node , ( 2 ) SCDCE : cross-entropy loss in the final node , and ( 3 ) SCDCEBNN : cross-entropy in the final node with binary weights throughout the model . Since sign activation is non-convex and our training starts from a different random initialization we run it a 100 times and output the majority vote . Algorithm 1 Stochastic coordinate descent for single hidden layer network Procedure : 1 . Initialize all network weights W , w to random values from the Normal distribution N ( 0 , 1 ) . 2 . Set network thresholds W0 to the median projection value on their corresponding weight vectors and w0 to the projection value that minimizes our network objective . while i < epochs do 1 . Randomly sample a batch of data equally from each class . ( We set this to 75 % of the training data in image and text data experiments and 25 % in the ECG data . ) 2 . Perform coordinate descent separately first on the final node w and then a randomly selected hidden node u ( a random column from the hidden layer weight matrix W ) 3 . Suppose we are performing coordinate descent on nodew . We select a random set of features ( coordinates ) from w called F . For each feature wi ∈ F we add/subtract a learning rate η and then determine the w0 that optimizes the loss ( done in parallel on a GPU ) . We consider all possible values of w0 = wT xi+w T xi+1 2 for i = 0 ... n− 2 and select the one that minimizes the loss ( also performed in parallel on a GPU ) . 4 . After making the update above we evaluate the loss on the full dataset ( performed on a GPU for parallel speedups ) and accept the change if it improves the loss . end while To illustrate our real runtimes and clean test accuracies we compare our models with a single hidden layer of 20 nodes to the equivalent network with sigmoid activation and logistic loss ( denoted as MLP ) and the binary neural network ( denoted as BNN ) Hubara et al . ( 2016 ) . We used the MLPClassifier in scikit-learn Pedregosa et al . ( 2011 ) to implement MLP and the Larq library Geiger & Team ( 2020 ) with the approx approximation to the sign activation . This has shown to achieve a higher test accuracy than the original straight through estimator ( STE ) of the sign activation Liu et al . ( 2018b ) . We perform a 1000 iterations of SCD01 and SCDCE and 10000 of SCDCEBNN . In Table 1 we show the runtimes of a single run of all models on CIFAR10 Krizhevsky ( 2009 ) ( 32× 32× 3 , 10K train , 2K test ) , CelebA facial attributes black hair vs brown hair Liu et al . ( 2015 ) ( 96×96×3 , 1K train , 1K test ) , GTSRB street sign recognition 60 vs 120 speed limit signs Stallkamp et al . ( 2011 ) ( 48×48×3 , 2816 train , 900 test ) , and ImageNet class 0 vs. 1 Russakovsky et al . ( 2015 ) ( 256 × 256 × 3 , 2580 train , 100 test ) . Our training runtimes are comparable to gradient descent in MLP and BNN and thus practically usable . We can trivially parallelize training an ensemble by doing multiple runs on CPU and GPU cores at the same time . We also show test accuracies of 100 vote ensembles of all models and find our model accuracies to be comparable to MLP and BNN . 3 RESULTS . Going forward we compare the adversarial robustness of ensembles of our three models SCD01 , SCDCE , and SCDCEBNN , their full precision and binary gradient descent trained equivalent counterparts MLP and BNN , two convolutional neural networks : LeNet LeCun et al . ( 1998 ) and ResNet50 He et al . ( 2016 ) , and random forests Breiman ( 2001 ) ( denoted as RF ) . For each model we use the majority vote output of 100 votes each with different initial parameters except for ResNet50 where we use 10 votes . In random forest we use an ensemble of 100 trees . We use a single hidden layer of 20 nodes in our three models and in MLP and BNN throughout the paper . The convolutional networks and random forest are not a fair comparison to our model since it has fewer parameters and does not perform bootstrapping or random feature selection as random forest . We include them nevertheless since convolutional neural networks serve as state of the art references and random forest serves as an alternative ensemble method .

Summary: The paper proposes an architecture (ensemble of networks) aiming at being robust against black-box attacks, based on the idea that crafting an adversarial example able to fool enough individual networks such that the majority vote changes is a more difficult task.  The paper presents ways of training such ensembles and provides several sets of experiments showing the advantage of the approach. It also contains an observation on "non-transferability", counting how many co-networks are fooled when only one is targetted by the blackbox attack. It turns out that this amount is lower for the proposed scheme.