# RobustQuote: Using Reference Images for Adversarial Robustness
## 4 Experiments
### Discussion  
The experiments in Section 4.2 showed that the gains of RobustQuote are conditioned on the asymmetric knowledge between the attacker and the defender, regarding the references. As stated in Section 4.1, the standard robust scenario occurs when the attacker knows everything about the model, the weights \(F_{\theta}\), and the reference dataset \(R_{\omega}\), but ignores the exact images \(I_{\omega}^{\text{R}}\). We stated in Section 3.6 the requirements on \(\mathcal{R}_{\omega}\) to render improbable the  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Block 4** & **Block 7** & **Block 10** & **Block 12** & **Natural** & **PGD-20** & **PGD-100** & **C\&W** & **AA** \\ \hline ✓ & & & & & 72.35 & 51.81 & 51.35 & 49.49 & 40.64 \\  & ✓ & & & & 78.54 & 59.34 & 60.10 & 59.87 & 47.26 \\  & & ✓ & & & 78.89 & 58.01 & 58.16 & 57.11 & 45.77 \\  & & & ✓ & & 74.71 & 59.67 & 59.69 & 63.93 & 41.01 \\ \hline ✓ & ✓ & & & & 72.01 & 50.81 & 50.79 & 48.05 & 39.92 \\  & ✓ & ✓ & & & 77.42 & 56.01 & 55.71 & 54.80 & 45.89 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Evaluation of the impact of various positions of our RobustQuote block within a network. The results are computed on a DeiT3-T backbone using the CIFAR10 dataset.  
random selection of \(I_{\omega}^{\rm R}\) prior to the inference time. We presented the hypothesis of a model deployed as a backend process, alongside which the defender has their own dataset \(R_{\omega}\) to generate \(I_{\omega}^{\rm R}\) in the backend too. When the user or the attacker uploads their input image \(I^{X}\), they will have no control or information on the references used during their inference. Only in the case where the user gains control over the references or when they can predict \(I_{\omega}^{\rm R}\) used at inference, can they generate the attacks with knowledge presented in Figure 4. In this case, our method is indeed neutralized and is then equivalent to the static adversarial training methods such as ARD-PRM. We believe that it is a reasonable defense strategy, as obtaining this level of control or information goes beyond the security of AI and now depends on the robustness of the overall design of the AI system deployed.  
Finally, we address the computational costs of our method. Our module, added to the original backbone of DeiT3-T, represents an increase of 11% of the model's weights. The set of references \(I_{\omega}^{\rm R}\) across the transformer block has a negligible impact in the training phase, as no gradient is tracked in the ViT blocks. In the inference stage, they occupy \(\frac{\omega}{R}\times 100\%\) additional memory alongside what \(I^{X}\) uses. In the RobustQuote block, \(A_{cls}\) has a complexity of \(O(\omega)\) and \(A_{rst}\) has a complexity of \(O(\omega N^{2})\). Compared to a standard ViT block with complexity of \(O(N^{2})\), this is an increase of \(\omega\times 100\%\). As such, in the context of CIFAR10 and ImageNette with 10 classes, this represents the memory footprint of 10 standard ViT blocks in parallel. On a system with Nvidia V100-16 GB GPU, using a batch size of \(B=64\) with DeiT3-T and CIFAR10 images, the standard DeiT3-T computes an inference in 66 ms using 29.3 MB of VRAM, while our RobustQuote requires 170 ms and uses 64.7 MB. This highlights the limitation of datasets with a small number of classes \(\omega\). A possible improvement would be to find a class agnostic quotation and rectification system, with a fixed number of references, providing a natural anchor instead of a class-specific anchor. We consider such improvement as future work.