# Segment Shards: Cross-Prompt Adversarial Attacks against the Segment Anything Model
## 7 Conclusions  
In this paper, we propose a method (the PBA method) to attack the significant visual foundation model (SAM), indicating that SAM has room for improvement in adversarial robustness. The experimental results demonstrate that the PBA method can successfully generate adversarial examples that perform well in both cross-prompt transferability and attack success rates. Numerical results on multiple datasets show that the cross-prompt attack success rate (\(ASR^{*}\)) of the PBA method is 17.83% higher on average, and the attack success rate (\(ASR\)) is 20.84% higher. Generating adversarial examples with prompt batching can effectively promote the cross-prompt transferability of adversarial examples. Additionally, we find that enhancing the cross-prompt transferability of adversarial examples is crucial for attacking visual foundation models equipped with the prompt mechanism.  
Additionally, we believe that using adversarial examples generated by the PBA method could be risky for real-world systems based on SAM. On the one hand, the PBA method exhibits strong cross-prompt transferability. On the other hand, SAM is a widely used visual foundation model. Therefore, we recommend adopting adversarial training when using SAM. This involves incorporating adversarial examples into SAM's training dataset to enhance its security.  
In future research, we will extensively focus on adversarial attacks and defenses targeting various types of foundation models, and explore defense methods against adversarial attacks in different application scenarios.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \(\mu\) & \(\mu\)**0.05** & \(\mu\)**0.1** & \(\mu\)**0.15** & \(\mu\)**0.2** & \(\mu\)**0.25** & \(\mu\)**0.3** & \(\mu\)**0.35** \\ \hline \multirow{2}{*}{PBA} & SSIM & 0.8606 & 0.8265 & 0.7818 & 0.7358 & 0.6981 & 0.6723 & 0.6583 \\ \cline{2-9}  & mIoU & 0.5854 & 0.3119 & 0.2098 & 0.1761 & 0.155 & 0.1503 & 0.1444 \\ \hline \multirow{2}{*}{PA} & SSIM & 0.8604 & 0.8258 & 0.7804 & 0.7351 & 0.6990 & 0.6746 & 0.6614 \\ \cline{2-9}  & mIoU & 0.6143 & 0.3936 & 0.2917 & 0.2405 & 0.2187 & 0.2022 & 0.1972 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Comparison results on SSIM and mIoU metrics between the adversarial examples generated by the PBA and PA methods. The \(\mu\) hyperparameter represents the maximum perturbation value that can be added during the adversarial example generation process.  
Figure 6: Illustration of the relationship between mIoU and SSIM under the PBA and PA methods. The dataset utilized in this experiment is CBCL300, with the range of variation for the maximum perturbation value, \(\mu\), ranging from 0.05 to 0.35.  
**Author Contributions:** Methodology, Q.F.; Software, Q.F.; Validation, Z.Z. and X.L.; Data curation, Z.Z. and J.Q.; Writing--original draft, Q.F.; Writing--review & editing, S.H., X.L. and G.S.; Visualization, G.S. and J.Q.; Funding acquisition, S.H. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research was funded by Natural Science Foundation of Chongqing, China: CSTB2022 NSCQ-MSX1454.  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** Publicly available datasets were analyzed in this study. These data can be found here: CBCL300, CH200, UNIMIB200. The code of this paper can be found in SegmentShards.  
**Conflicts of Interest:** The authors declare no conflicts of interest.