Question: Why is the BaB attack speed a little bit slow?
Evidence: Setup. We evaluate our methods on all 10,000 test examples of and CI-FAR10 dataset, and select 9 models which are mostly benchmarking models used in previous works or competitions (details in Appendix A). For each model, we first run three commonly used strong adversarial attacks: a multi-targeted PGD (MT-PGD) attack with 100 Adam steps and 30 random restarts; a multi-targeted PGD attack with Output Diversified Sampling (ODS-PGD) with 100 Adam steps and 30 random restarts, and AutoAttack which is an ensemble of parameter-free attacks.
The remaining robust images are then tested with α,β-CROWN verifier to see if they can be verified robust so no further attack is needed. Any images that failed with the verifier are then evaluated in an MIP formulation  solved using Gurobi. The MIP solver is used as the last resort because it is usually much more expensive than other approaches. We set an one hour timeout for MIP attack and our BaB attack, but our attack usually terminates much faster than a generic MIP solver. Any instance that cannot be attacked via a combination of MT-PGD, ODS-PGD and AutoAttack nor verified is referred to as a hard instance, and the main evaluation is conducted on these instances. All attacks are ∞ norm-based with listed in Table. The hyperparameters can be found in Appendix A.2. Results. Table shows a breakdown of all 10,000 test examples on the 5 models. Among these models, MNIST Small-Adv, MNIST Conv-Small, CIFAR LP D -CNN-A, and LP D -ResNet are relatively easy, and most of datapoints can be either attacked or verified by SOTA tools. The LP D -CNN-A and LP D -ResNet models were among the largest ones in , however they have become quite easy due to the recent progress of neural network verifiers -only 2 and 1 (out of 10,000) examples are hard instances. On the relatively small MNIST Small-Adv model, a MIP solver can terminate within a reasonable amount of time, and we can attack the same number of images while being faster.
On the MNIST CNN-A-Adv model, many instances timeout with the MIP solver and we can attack more images under less time; see Fig. and Fig. (in Appendix) for time vs. number of images attacked. The CIFAR CNN-B-Adv, CNN-A-Mix, and Marabou models are much harder, and we managed to solve 5, 17, and 27 more images compared to MIP attack within much shorter time.
BaB attack can solve the same number of images as MIP attack on MNIST Conv-Small and CIFAR CNN-A-Adv but is over 10x faster in average. For images that none of the attacks work, we plot the minimum margin between the ground-truth label and other labels in Fig. and compare the margins against ODS-PGD, MT-PGD attacks and AutoAttack. Our method consistently achieves smaller margins on all 6 models.
Comparison to more attacks. We compare our BaB attack to 5 state-of-the-art attacks on all hard instances in Table. Attacks we evaluated include 1) FAB attack): a whitebox boundary adversarial attack; 2) Square attack: a query-efficient black-box adversarial attack using randomized search with localized square-shaped updates; 3) DIFGSM attack: a white box attack leveraging input diversity; 4) MIFGSM attack: a momentum-based iterative white box attack to escape from poor local maxima and 5) Distributional attack: a white box attack considering adversarialdata distribution. We increased steps and iterations to make some attacks stronger; hyperparameters are in Appendix A.3. Most attacks are not effective on these hard instances. Square attack is the strongest among them but still finds less adversarial examples than us and is also relatively slow.
Ablation study. To fully understand how each proposed component contributes to the overall performance of BaB attack, we conduct ablation study on MNIST Small-Adv and MNIST CNN-A-Adv models and show the number of successfully attacked instances and average time in Table. We first show that the beam search guided by verifier α,β-CROWN significantly helps the performance compared to random top-down search without any guidance. We then show that the top-down diving and bottom-up large neighborhood search contribute to different hard instances and the combination of them lead to the best attack performance. Note that the average time here is on successfully attacked samples by each method only, showing the efficiency of each component instead of strength.
Adversarial examples and attacks. Adversarial examples were first discovered in and they can be easily constructed by single-/multi-step gradient descent to fool regularly trained neural networks. However, purely gradient-based methods can fail due to gradient masking and obfuscated gradients. Popular attacks like PGD or CW 2018;  despite their empirically good performance and efficiency. A large body of white-box adversarial attacks have been proposed to strengthen adversarial attacks; many of them are variants of PGD based attacks. Due to non-convexity of the adversarial attack objective, gradient-free methods and black-box attacks are also widely explored but mostly result in similar or worse performance compared to gradient-based ones. Recently, stronger attacks are proposed but they are also restricted to searching in input space. In this paper, we are the first to conduct a systematic and efficient search of adversarial examples in activation space inspired by branch and bound techniques used for integer optimization and neural network verification.
Neural network verification. Early neural network verifiers solve the verification problem with satisfiability modulo theories (SMT) or MIP solvers and can only scale to very small networks. Efficient verification methods with various sound relaxations are then proposed for verifying larger networks but without completeness guarantee. Branch and bound (BaB) based verifiers can efficiently branch on ReLU neurons and achieve completeness on ReLU networks using efficient incomplete verifiers. BaB based methods with input domain split and refinements are also investigated but they are limited to low input dimensions. Recent verifiers use GPU accelerated incomplete solvers to further scale up verification and achieve several orders of magnitude speedup. Our BaB attack relies on the lower bounds of BaB subdomains computed by α,β-CROWN, one of the most efficient GPU-based verifiers that won the second neural network verification competition (VNN-COMP 2021).
Table 1: Comparison between MIP attack (Tjeng et al., 2019) (using Gurobi) and our BaB Attack. Both attacks focus on hard instances where a combination of MT-PGD, ODS-PGD and AutoAttack cannot attack and their robustness also cannot be verified using a verifier. We are faster than MIP Attack and can also find more adversarial examples in all models. Average time excludes examples where all methods time out.****Table 2: Number of successfully (# succ.) attacked hard i~nstances (from Table 1) under five more attacks.
Answer: We do agree our attack is relatively slow because we conduct a systematic search on activation space. It should be used as a last resort after existing fast attack algorithms searching directly in input space (e.g., PGD) fail. In Table 2, we show that we can attack more images and discover adversarial examples where all existing adversarial attacks fail. Also, compared to some other strong attacks such as FAB and Square attacks, our time is similar to theirs (see new results in Table 2). Additionally, we are also significantly faster (>10x on most models) than the MIP based attack which is also an exhaustive attack on activation space like ours (Table 1).