# RobustQuote: Using Reference Images for Adversarial Robustness
## 4 Experiments
### Robustness Evaluation  
We first evaluated the effectiveness of RobustQuote in improving robustness against adversarial attacks on DeiT3 backbones. The accuracy scores are summarized in Table 1. RobustQuote consistently improved accuracy under adversarial perturbation of DeiT3 backbones. We also observed a good natural accuracy, unlike defense methods such as FSR or SACNet, which degraded it by up to 4.5%. RobustQuote's performance remained within 2.5% of the best-performing ARD+PRM at 78.54% on CIFAR10 and 82.73% on ImageNette. More interestingly, we observed a drastic increase in adversarial robustness of +7.7% on average on CIFAR10 and +4.3% on ImageNette compared to the best-performing ARD+PRM. We also evaluated the evolution of adversarial robustness as the attack budget increases. Although the attack budget increased from PGD-20 to PGD-100 to C&W, which are increasingly more efficient at reducing accuracy on the other models, we observed almost identical accuracy (within 0.8%) with RobustQuote. Only AutoAttack produced a substantial accuracy drop of 12.9% on CIFAR10 and 7.5% on ImageNette. This is because the AutoAttack implementation requires multiple attacks on each sample, triggering, by design, a new set of reference images. This leads AutoAttack to look for the best attack on the weakest set of references. We explain the behavior of the previous PGDs and C&W attacks by arguing that the better an attack is at optimizing a perturbation for a given \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Attacks** & **DeiT3-T** & **ARD+PRM** & **FSR** & **SACNet** & **DH AT** & **Ours** \\ \hline \hline
**\# Params** & 5.4M & 5.4M & 5.4M & 6M & 12M & 6M \\ \hline \hline
**CIFAR10** & & & & & & \\ \hline Natural & 79.41 & **80.57** & 75.71 & 77.04 & 79.81 & 78.54 \\ PGD-20 & 49.41 & 50.52 & 42.75 & 47.89 & 50.41 & **59.34** \\ PGD-100 & 49.38 & 50.46 & 42.68 & 47.81 & 50.37 & **60.10** \\ C\&W-20 & 46.19 & 47.62 & 39.08 & 45.22 & 47.09 & **59.87** \\ AutoAttack & 45.56 & 46.90 & 38.32 & 44.42 & 46.40 & **47.26** \\ \hline \hline
**ImageNette** & & & & & & \\ \hline Natural & 82.42 & **85.32** & 79.06 & 82.45 & 83.41 & 82.73 \\ PGD-20 & 56.25 & 56.51 & 49.04 & 55.54 & 56.54 & **62.55** \\ PGD-100 & 56.08 & 56.28 & 48.79 & 55.16 & 56.43 & **61.89** \\ C\&W & 55.75 & 56.69 & 45.48 & 55.01 & 56.25 & **61.71** \\ AutoAttack & 54.70 & 55.46 & 44.69 & 53.76 & 55.46 & **55.97** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Robustness to adversarial attacks of state-of-the-art adversarial defenses and our RobustQuote-7. We measure the accuracy (in %) and report the best (higher the better) results in **bold**. input, the worse this perturbation will perform in a new setting with the same input but with different references. In this paradigm, attacks that come closer to the definition of a universal perturbation will perform better than specific attacks. Evaluation for Black-Box and Adaptive Attacks  
We further investigated various attack scenarios to understand the robustness conditions of our RobustQuote model. First, we report in Figure 4 the scenario in which the adversary possesses perfect knowledge of the inference time. In this situation, the benefits of using reference images are completely lost, and the attacker knows in advance which reference images will be used during the inference. We observed a notable decrease in accuracy (ranging from 3.8% to 12.8%) under PGD and C&W, where RobustQuote previously exhibited a significant advantage over existing methods. In this context, RobustQuote remained within 0.8% of the original DeiT3, proving that additional evasions are not introduced with references. In view of the aforementioned behaviors, one possible explanation of RobustQuote's robustness is that it performs some kind of gradient obfuscation [10].