# Watermarking Tiny MLCommons Image Applications Without Extra Deployability Costs
## 9 Attacks and Results  
This section reports the experimental analysis of the resilience of the watermarked models through a series of simulated attacks. The goal is to evaluate how well these models can withstand adversarial conditions without prior knowledge of the model's inner workings. As outlined in Sections 6.1 and 6.2, both white-box and black-box watermark \begin{table}
\begin{tabular}{c c c} \hline \hline
**Available Resources** & **STM32H743 ZI MCU** & **x86 Host** \\ \hline RAM & 1 MB & 16 GB \\ \hline FLASH & 2 MB & 1 TB \\ \hline CPU frequency & 480 MHz & i5 CPU, 1.60 GHz \\ \hline Core number & 1 & 8 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Comparison of available resources provided by an MCU and the x86 CPU. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{3}{c}{**MobileNetV1 (FP32)**} & \multicolumn{3}{c}{**ResNet-V1 (FP32)**} \\ \cline{2-6}
**Board Name** & **Latency** & **RAM** & **FLASH** & **Latency** & **RAM** & **FLASH** [KiB]** \\ \hline NUCLEO-H743 ZI2 @480 MHz & 110.6 & 167 & 855 & 122.5 & 140 & 323 \\ \hline STM32L4R9I-DISCO @120 MHz & 835.9 & 167 & 855 & 1.033 & 140 & 323 \\ \hline B-U585I-IOT02A @160 MHz & 454.8 & 167 & 855 & 528.6 & 140 & 323 \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Results achieved on the MCU memory footprint and inference time, measured with ST Edge AI Core Technology, for the models explained in Sections 5.1 and 5.2, respectively. ing techniques were implemented, each with a distinct embedding or detection strategy. The upcoming sections will focus on these models, subjecting them to a variety of simulated attack scenarios, including Gaussian noise injection, pruning, and quantization. These simulations are designed to test the robustness and security of the watermarks, providing insight into the potential vulnerabilities of these models under adversarial stress. The MPAI ([[https://mpai.community/](https://mpai.community/)]([https://mpai.community/](https://mpai.community/)) accessed on 13 November 2024) (Moving Picture, Audio and Data Coding by Artificial Intelligence) community established the IEEE 3304-2023 standard [52] for NNW to ensure the robustness and reliability of AI models. According to this standard, NN models may be subjected to various types of attacks to evaluate their resilience and performance. This is crucial for maintaining the integrity and reliability of AI systems in real-world applications. The models were tested under three different types of attacks:  
* Gaussian involves adding random noise to the data, which can potentially disrupt the model's ability to output accurate predictions. Gaussian noise is a common form of statistical noise that, when it affects the input data, leads to degraded performance. For the attacks, the values 0.001, 0.01, 0.1, 1, and 10 represented the standard deviation of the added noise, with higher numbers indicating more noise that can disrupt the model's performance more and potentially its watermark;
* Pruning selectively removes connections between neurons, effectively reducing the complexity of the NN and potentially degrading its performance. Pruning is often used to simplify a model's complexity, but in this context, it is used to test the model's robustness. In these attacks, the numbers 0.1 to 0.5 indicated the fraction of the model's weights that were removed, with higher values representing more aggressive pruning that can degrade both the model's accuracy and the integrity of the watermark;
* Quantization involves reducing the precision of numerical values, which can lead to a loss of information and accuracy. Quantization typically reduces the bit depth of the weights of the NN, and according to the MPAI documentation, this is applied only to the weights as fake quantization. The reduction in bit depth can significantly impact the model's performance.