Sparse Black-Box Multimodal Attack for Vision-Language Adversary Generation (2023)

Abstract: Deep neural networks have been widely applied in real-world scenarios, such as product restrictions on e-commerce and hate speech monitoring on social media, to ensure secure governance of various platforms. However, illegal merchants often deceive the detection models by adding large-scale perturbations to prohibited products, so as to earn illegal profits. Current adversarial attacks using imperceptible perturbations encounter challenges in simulating such adversarial behavior and evaluating the vulnerabilities of detection models to such perturbations. To address this issue, we propose a novel black-box multimodal attack, termed Sparse Multimodal Attack (SparseMA), which leverages sparse perturbations to simulate the adversarial behavior exhibited by illegal merchants in the black-box scenario. Moreover, SparseMA bridges the gap between images and texts by treating the separated image patches and text words uniformly in the discrete space. Extensive experiments demonstrate that SparseMA can identify the vulnerability of the model to different modalities, outperforming existing multimodal attacks and unimodal attacks. SparseMA, which is the first proposed method for black-box multimodal attacks to our knowledge, would be used as an effective tool for evaluating the robustness of multimodal models to different modalities.

## introduction with the rapid development of deep neural networks (dnns), vision-language multimodal classification has been applied in various real-world applications, such as product restrictions on ecommerce @xcite and hate speech monitoring on social media @xcite , to ensure secure governance of various platforms. however, there are always some illegal merchants who attempt to sell prohibited goods to make illegal profits, such as illegal drugs or smuggled goods, which violate the rules and terms of platforms. due to the lack of professional knowledge in adversarial attacks @xcite , they often adopt large-scale and sparse perturbations, including smearing and mosaicing, to deceive detection models deployed by various companies, thereby posing a significant threat to the security of platforms, as shown in figure 1 . current adversarial attacks @xcite typically mislead the victim model by adding imperceptible perturbations to benign samples, which can not accurately evaluate the vulnerability of detection models to the adversarial behavior exhibited by illegal merchants. in addition, prior research has mainly focused on unimodal adversarial attacks in computer vision (cv) @xcite and natural language processing (nlp) @xcite , with little attention paid to the vulnerability of multimodal models @xcite that are more challenging but widely used in the real world. this motivates us to develop a multimodal adversarial attack that simulates physical adversarial behaviors in the black-box scenario, which can help ensure secure governance of various platforms. existing multimodal attacks @xcite typically process images and texts independently due to the vastly different data attributes of the two modalities, such as continuous images vs. discrete texts. when employing different attack strategies to perturb images and texts, these methods are unable to effectively combine features from different input modalities, leading to relatively low attack performance. and these attacks run in white-box attack settings, which are almost impossible to apply to the real-world. to address the above issue and simulate the sparse perturbations commonly used by illegal merchants, we propose to map them into the same discrete space to bridge the gap between continuous images and discrete texts. and then we perturb both images and texts simultaneously using the same sparse attack strategy. although these sparse perturbations are indeed visible, they generally do not alter the semantics. in this work, we propose a novel black-box multimodal attack, named sparse multimodal attack (sparsema). sparsema splits the input image into multiple patches, then evaluates the impact of each patch in the image and each word in the input text on the victim model based on the output logits. after sorting all patches and words according to their impacts, we sequentially replace the original data with suitable candidates until an adversarial example is found. note that sparsema only needs access to the model output, making it more feasible in the real-world than white-box attacks, which require full access to the model, including architecture, parameter, gradient, output, etc. to validate the effectiveness of the proposed sparsema, we do comparison with one image attack, two text attacks, and two multimodal attacks on the task of vision-language multimodal classification. extensive experiments demonstrate that sparsema could identify the vulnerability of the model to different modalities, achieving a higher attack success rate than almost all the baselines. moreover, sparsema could generate better sparse adversarial examples with fewer perturbations, making them more similar to benign samples and more applicable to real-world scenarios. through analyzing the impact of perturbations on the model's output in each modality, sparsema can reveal the relative importance of each modal-ity in the model's decision-making process. this information is valuable for researchers to better understand the multimodal model's behavior and enhance its robustness against adversarial attacks. ## related work this section provides a brief overview of unimodal adversarial attacks on images or texts, as well as multimodal adversarial attacks. ## methodology in this section, we first formalize the problem of multimodal adversarial attack and then provide a detailed description of our method. ## experiments in this section, we conduct extensive experiments to validate the effectiveness of sparsema on three datasets and three models that are widely used. ## conclusion we propose a vision-language black-box adversarial attack method called sparse multimodal attack (sparsema). sparsema maps the input image and text into discrete space by splitting the image into patches and the text into words. then it evaluates the importance of each patch or word based on its affects to the model's output. by greedily substituting important patches or words with suitable candidates, sparsema generates adversarial examples with fewer perturbations and higher quality, which achieves a higher attack success rate than the baselines. additionally, sparsema can reveal the vulnerability of multimodal models to different modalities and concentrate on perturbing the more vulnerable ones. it could be served as a good metric to measure the vulnerability of multimodal models to different modalities. and it can help us better understand the multimodal model's behavior, and enhance its robustness against adversarial attacks. experiments show that the sparse perturbations generated by sparsema are more practical in the physical world that are easier to be added for attackers, and perform well against real-world influences. sparsema would be a strong baseline for future works and may inspire more researches on multimodal attacks.