SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This research is partly supported by ARO Grant W911NF1810295, ARO Grant W911NF2210010, NSF IIS-1909644, NSF CMMI-2024688 and NSF IUSE-2013451. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ARO, NSF, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.
Akshay Agrawal, Brandon Amos, Shane Barratt, Stephen Boyd, Steven Diamond, and J Zico Kolter. Differentiable convex optimization layers. Advances in neural information processing systems, 32, 2019.
Ahmed Aldahdooh, Wassim Hamidouche, Sid Ahmed Fezza, and Olivier Déforges. Adversarial example detection for dnn models: A review and experimental comparison. Artificial Intelligence Review, 55(6):4403-4462, 2022.
Brandon Amos and J Zico Kolter. Optnet: Differentiable optimization as a layer in neural networks. In International Conference on Machine Learning, pages 136-145. PMLR, 2017.
Andrew Butler and Roy H Kwon. Efficient differentiable quadratic programming layers: an admm approach. Computational Optimization and Applications, 84(2):449-476, 2023.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017, IEEE symposium on security and privacy (sp), pages 39-57. IEEE, 2017.
Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Chau. Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2018, Dublin, Ireland, September 10–14, 2018, Proceedings, Part I 18, pages 52–68. Springer, 2019.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pages 1310-1320. PMLR, 2019.
MMPreTrain Contributors. Openmmlab's pre-training toolbox and benchmark. https://github.com/open-mmlab/mmpretrain, 2023.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9185-9193, 2018.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invariant attacks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 4312-4321, 2019. URL http://openaccess.thecvf.com/content_CVPR_2019/html/Dong_Evading_Defenses_to_Transferable_Adversarial_Examples_by_Translation-Invariant_Attacks_CVPR_2019_paper.html.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 2: Short Papers, pages 31-36, 2018. doi: 10.18653/v1/P18-2006. URL https://www.aclweb.org/anthology/P18-2006/.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille. Improving transferability of adversarial examples with input diversity. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 2730-2739, 2019b. URL http://openaccess.thecvf.com/content_CVPR_2019/html/Xie_Improving_Transferability_of_Adversarial_Examples_With_Input_Diversity_CVPR_2019_paper.html.
Chaoning Zhang, Philipp Benz, Adil Karjauv, Jae Won Cho, Kang Zhang, and In So Kweon. Investigating top-k white-box and transferable black-box attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15085-15094, 2022.
Zekun Zhang and Tianfu Wu. Learning ordered top-k adversarial attacks via adversarial distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 776-777, 2020.
The selection of learning rates  $\gamma$  and  $\lambda$  values (see Eqn. 10) in the context of learning attacks requires careful consideration to achieve the desired trade-offs (see Fig. 3) and optimize the attack performance. In this regard, the learning rate is adapted based on the value of the number of targets,  $K$ . Empirically, we observed that as  $K$  increases, a higher learning rate tends to yield a better balance between Attack Success Rate (ASR) and perturbation energy consumption. Specifically, for different ranges of  $K$ , the learning rate is adjusted accordingly, ensuring an appropriate scaling factor to guide the optimization process. Additionally, we find convolutional models require a larger learning rate than Transformer models to reach desirable attacks in all tested methods.
- If  $K < 5$ , the learning rate is set to  $\gamma = 0.75e - 3$  (for all the four models).
- If  $5 \leq K < 10$ , the learning rate is set to  $\gamma = 1.0e - 3$  (for ViT-B and DEiT-S) and  $\gamma = 2.0e - 3$  (for ResNet-50 and DenseNet-121).
- If  $10 \leq K < 15$ , the learning rate is set to  $\gamma = 1.0e - 3$  (for ViT-B and DEiT-S) and  $\gamma = 3e - 3$  (for ResNet-50 and DenseNet-121).
- If  $15 \leq K < 20$ , the learning rate is set to  $\gamma = 1.5e - 3$  (for ViT-B and DEiT-S) and  $\gamma = 3.5e - 3$  (for ResNet-50 and DenseNet-121).
- If  $K \geq 20$ , the learning rate is set to  $\gamma = 2e - 3$  (for ViT-B and DEiT-S) and  $\gamma = 4e - 3$  (for ResNet-50 and DenseNet-121).
Similarly, the choice of the weight parameter  $\lambda$  in the loss function (Eqn. 10) also plays a crucial role. We use  $\lambda$  to weight the first term in both Eqn. 3 and Eqn. 10 (the loss term that finds a successful attack when optimized) and we leave the second term (the energy penalty) unweighted. In challenging attacks where  $K \geq 5$ , the range of suitable  $\lambda$  values that achieve desirable trade-offs between ASR and energy is significantly wider compared to easier attacks ( $K = 1$ ). Given the multitude of appropriate  $\lambda$  values for difficult attacks, we do not perform explicit tuning of  $\lambda$  in computing results in Table 1, since different choices of  $\lambda$  would correspond to different points along the energy/ASR trade-off curve (which are used in generating the trade-off curves in Fig 3).
For attacks with  $K = 1$ , selecting an excessively high  $\lambda$  can lead to inefficient energy usage. For instance, consider an attack with  $\lambda = 5$  achieving an ASR of 1.0 and an energy cost of 2.0, while increasing  $\lambda$  to 10 maintains the ASR at 1.0 but raises the energy cost to 5.0. In this case, the choice of  $\lambda = 5$  is preferable as it achieves the desired ASR with lower energy consumption.
For our QuadAttac  $K$  which operates in a high-dimensional latent space with much higher loss magnitudes than  $\mathrm{CW}^K$  and  $AD$  (which operate in the logit or probability space), a lower value of  $\lambda$  is necessary to reach the optimal point for  $K = 1$ . Hence, we set  $\lambda = 0.5$  for QuadAttac  $K$  and  $\lambda = 5$  for the logit/probability space losses for the  $K = 1$  case. For all other values of  $K$ , we use  $\lambda = 10$  since these attacks are more challenging and require a higher weight on the top-  $K$  term to attain the desired ASR and none of the tested methods reach ASR saturation points on  $K >= 5$  for the chosen  $\lambda$ .
B More Qualitative Results
In addition to quantitative evaluations in Table 1, we provide detailed visualizations and qualitative analysis to gain deeper insights into the behavior and impact of our QuadAttac  $K$  method on the classification models. These visualizations offer a comprehensive understanding of the attack process, showcasing the changes in both image perturbations and attention maps (for Transformer models). By examining the visual patterns and comparing the distributions of clean and attacked class predictions, we can explore the effects of our QuadAttac  $K$  on the attacked models' predictions and gain valuable insights into the robustness of these models. These visual analyses serve as a valuable complement to our quantitative assessments, providing a holistic perspective on the performance and behavior of our QuadAttac  $K$  across various attack scenarios.
Furthermore, it is important to note that while our QuadAttac  $K$  does not prioritize maximizing the margins between class probabilities, its quadratic programming approach fundamentally enables the
Clean Image & Attn
QuadAttacK
Figure 4: Examples of attacking DeIT-S [Touvron et al., 2021] with  $K = 5$ : the 1st and 2nd rows show results for  $CW^K$  and Adversarial Distillation (AD) [Zhang and Wu, 2020] respectively, and the 3rd row shows results for QuadAttacK. The ground-truth label is Weimaraner. The ordered top-5 targets (randomly sampled and kept unchanged for the different attack methods) are: [table lamp, langur, wig, hip, piggy bank], while for example the original top-5 predictions of DEiT-S are [Weimaraner, Bedlington terrier, German short-haired pointer, Great Dane, Yorkshire terrier].
integration of such constraints if desired for other applications. The flexibility of our method allows for the incorporation of additional objectives or constraints that prioritize class separability, enabling researchers to tailor the optimization process according to specific needs. This versatility opens up avenues for exploring variations of the QuadAttac  $K$  framework and adapting it to diverse scenarios where increasing the margins between class probabilities is a desirable objective.
Adv. Example
A D
Figure 5: Examples of attacking DEiT-S [Touvron et al., 2021] with  $K = 10$ : the 1st row shows results for Adversarial Distillation (AD) [Zhang and Wu, 2020], and the 2nd row shows results for our QuadAttacK, and  $\mathrm{CW}^K$  fails for this example. The ground-truth label is Weimaraner. The ordered top-10 targets (randomly sampled and kept unchanged for the different attack methods) are: [table lamp, langur, wig, hip, piggy bank, American Staffordshire terrier, school bus, crossword puzzle, entertainment center, ibex], while for example the original top-10 predictions of DEiT-S are [Weimaraner, Bedlington terrier, German short-haired pointer, Great Dane, Yorkshire terrier, Siamese cat, butcher shop, silky terrier, Italian greyhound, vizsla].
aannnnnne
(a) Clean Image
(b) Adversarial Image
(c) Adversarial Perturbation
(d) Clean Image's Class Prob.
Figure 6: Top-15 attack example 1/2 on ResNet-50 [He et al., 2016] with QuadAttacK (ours). The original Top-15 predictions are [beaver, mink, otter, weasel, platypus, porcupine, American coot, water snake, red-breasted merganser, guinea pig, sea lion, red-backed sandpiper, European gallinule, little blue heron, limpkin ]. The ordered Top-15 targets (randomly sampled) are: [oystercatcher, desktop computer, shovel, bib, crane, spatula, torch, acorn, dalmatian, cheetah, gown, bull mastiff, microwave, cornet, puffer ].
(e) Adversarial Image's Class Prob.
(d) Clean Image's Class Prob.
Figure 7: Top-15 attack example 1/2 on DenseNet-121 [Huang et al., 2017] with QuadAttacK (ours). The original Top-15 predictions are [ chimpanzee, gorilla, siamang, orangutan, ribbon, sloth bear, colobus, howler monkey, guenon, American black bear, spider monkey, baboon, macaque, patas, water buffalo ]. The ordered Top-15 targets (randomly sampled) are: [ window shade, garden spider, cowboy hat, comic book, iPod, mortarboard, leaf beetle, stupa, zucchini, cock, beer bottle, cab, sunglass, fiddler crab, Windsor tie ].
(d) Clean Image's Attention
(e) Adversarial Image's Attention
(f) Clean Image's Class Prob.
Figure 8: Top-15 attack example 1/2 on DeiT-S [Touvron et al., 2021] with QuadAttacK (ours). The original Top-15 predictions are [ nematode, isopod, digital clock, jellyfish, crossword puzzle, flatworm, drumstick, jack-o'-lantern, knot, bassoon, safety pin, paper towel, thunder snake, matchstick, trombone ]. The ordered Top-15 targets (randomly sampled) are: [ axolotl, quail, hyena, carbonara, hen, oboe, mud turtle, robin, Italian greyhound, oystercatcher, space shuttle, airliner, Bedlington terrier, miniature pinscher, iron ].
(g) Adversarial Image's Class Prob.
(f) Clean Image's Class Prob.
Figure 9: Top-15 attack example 1/2 on ViT-B [Dosovitskiy et al., 2020] with QuadAttacK (ours). The original Top-15 predictions are [ Band Aid, airship, warplane, airliner, wing, mouse, space bar, face powder, missile, ballpoint, space shuttle, kite, modem, revolver, speedboat ]. The ordered Top-15 targets (randomly sampled) are: [ padlock, king snake, screwdriver, Welsh springer spaniel, bookshop, triumphal arch, shoe shop, Italian greyhound, diamondback, missile, drilling platform, worm fence, sea snake, African elephant, joystick ].
(d) Clean Image's Class Prob.
Figure 10: Top-10 attack example 1/2 on ResNet-50 [He et al., 2016] with QuadAttacK (ours). The original Top-10 predictions are [mink, beaver, hippopotamus, chimpanzee, weasel, otter, American alligator, mud turtle, Rottweiler, terrapin ]. The ordered Top-10 targets (randomly sampled) are: [ tiger cat, black-and-tan coonhound, dung beetle, hook, banded gecko, hognose snake, skunk, ashcan, patio, admiral ].
(d) Clean Image's Class Prob.
Figure 11: Top-10 attack example 1/2 on DenseNet-121 [Huang et al., 2017] with QuadAttacK (ours). The original Top-10 predictions are [ coyote, red wolf, grey fox, dhole, dingo, red fox, kit fox, lynx, timber wolf, hyena ]. The ordered Top-10 targets (randomly sampled) are: [ chain, hermit crab, tiger cat, sweatshirt, packet, European fire salamander, electric locomotive, mobile home, fiddler crab, car mirror ].
(f) Clean Image's Class Prob.
(g) Adversarial Image's Class Prob.
Figure 12: Top-10 attack example  $1 / 2$  on DeiT-S [Touvron et al., 2021] with QuadAttacK (ours). The original Top-10 predictions are [ wooden spoon, ladle, French horn, snowmobile, mortar, spatula, paddle, rocking chair, fly, frying pan ]. The ordered Top-10 targets (randomly sampled) are: [ tennis ball, langur, toy terrier, pool table, patio, ballpoint, bee eater, fireboat, toy poodle, soup bowl ].
(f) Clean Image's Class Prob.
Figure 13: Top-10 attack example  $1 / 2$  on ViT-B [Dosovitskiy et al., 2020] with QuadAttacK (ours). The original Top-10 predictions are [chocolate sauce, ice cream, trifle, pomegranate, bakery, fig, hay, strawberry, burrito, rapeseed]. The ordered Top-10 targets (randomly sampled) are: [ goblet, shopping cart, titi, maillot, racer, expressomaker, zebra, Shih-Tzu, hand blower, speedboat ].
(d) Clean Image's Class Prob.
Figure 14: Top-5 attack example  $1/2$  on ResNet-50 [He et al., 2016] with QuadAttacK (ours). The original Top-5 predictions are [Gordon setter, cocker spaniel, Irish setter, Sussex spaniel, English setter ]. The ordered Top-5 targets (randomly sampled) are: [indri, television, grasshopper, espresso maker, Yorkshire terrier ].
(d) Clean Image's Class Prob.
Figure 15: Top-5 attack example 1/2 on DenseNet-121 [Huang et al., 2017] with QuadAttacK (ours). The original Top-5 predictions are [Gordon setter, Yorkshire terrier, Bernese mountain dog, black-and-tan coonhound, Brabancon griffon ]. The ordered Top-5 targets (randomly sampled) are: [ indri, television, grasshopper, espresso maker, Yorkshire terrier ].
(f) Clean Image's Class Prob.
Figure 16: Top-5 attack example  $1/2$  on DeiT-S [Touvron et al., 2021] with QuadAttacK (ours). The original Top-5 predictions are [home theater, cinema, projector, theater curtain, pool table ]. The ordered Top-5 targets (randomly sampled) are: [tow truck, snow leopard, cairn, EntleBucher, Leonberg ].
(f) Clean Image's Class Prob.
Figure 17: Top-5 attack example 1/2 on ViT-B [Dosovitskiy et al., 2020] with QuadAttacK (ours). The original Top-5 predictions are [Band Aid, airship, warplane, airliner, wing]. The ordered Top-5 targets (randomly sampled) are: [padlock, king snake, screwdriver, Welsh springer spaniel, bookshop ].
(d) Clean Image's Class Prob.
Figure 18: Top-1 attack example  $\frac{1}{2}$  on ResNet-50 [He et al., 2016] with QuadAttacK (ours). The original Top-1 predictions are [ ashcan ]. The ordered Top-1 targets (randomly sampled) are: [ eggnog ].
(d) Clean Image's Class Prob.
Figure 19: Top-1 attack example  $1/2$  on DenseNet-121 [Huang et al., 2017] with QuadAttacK (ours). The original Top-1 predictions are [starfish ]. The ordered Top-1 targets (randomly sampled) are: [ crayfish ].
(f) Clean Image's Class Prob.
Figure 20: Top-1 attack example  $1/2$  on DeiT-S [Touvron et al., 2021] with QuadAttacK (ours). The original Top-1 predictions are [sandal]. The ordered Top-1 targets (randomly sampled) are: [espresso].
(f) Clean Image's Class Prob.
Figure 21: Top-1 attack example  $1/2$  on ViT-B [Dosovitskiy et al., 2020] with QuadAttacK (ours). The original Top-1 predictions are [bluetick ]. The ordered Top-1 targets (randomly sampled) are: [hen ].