# Watermarking Tiny MLCommons Image Applications Without Extra Deployability Costs
## 9 Attacks and Results
### Results for the White-Box Watermarking Method  
In this subsection, the resistance to attacks explained above will be presented for WMd and unWMd models with datasets explained in Sections 5.1 and 5.2 considering the watermarking algorithm explained in Section 6.1 with the form of the watermark matrix in Section 6.1.2 set as direct. In addition to the results from the inferences with the original dataset as input, the BER extracted value for each model is reported in Tables 13 and 14. To ensure the correct identification of watermarks, a BER threshold of 0% has been adopted. In other words, a watermark was considered to be correctly recognized only if each single extracted bit matched exactly the original inserted bit. Results that met this criterion are highlighted in green.  
The results of the experiments discussed in Section 7.1.1 and reported in Table 13 reveal varying degrees of effectiveness for each attack on the watermarked model that uses the dataset explained in Section 5.1 that uses a MobilenetV1 model with the Sections 6.1 and 6.1.2 approaches in direct mode. It has to be noted that when the model is subject to the three types of attacks (Gaussian, prune, and quantization, its performance in terms of accuracy shows a decreasing trend as the attack power increases, in fact, from the lowest values of attack an accuracy that goes from 71.66% to 60.93% could be seen. Despite the decrease in accuracy values, the model remains robust in terms of the watermark detected; in fact, it has been possible to notice that the BER value remained steady at 0% in all attack cases except for the Gaussian attack with an attack value of 10, in which the BER is 8.98%.  
The results of experiments discussed in Section 7.1.2 and reported in Table 14 revealed varying degrees of effectiveness for each attack on the watermarked model that uses the dataset explained in Section 5.2 that uses a customized ResNet-8 with the Sections 6.1 and 6.1.2 approaches in direct mode. In contrast to the results presented in Table 13, the model showed greater resistance to attacks of lower intensity. In fact, an accuracy above 70% can be seen in the first three levels of attack for each case (Gaussian, prune, and quantization). However, when exposed to more powerful attacks, the model's performance dropped significantly. Regarding the detection of the watermark, behavior similar to that described in Table 13 was observed, with the exception of the Gaussian attack with an attack value of 10, in which a BER of 0.26% was obtained.  
\begin{table}
\begin{tabular}{l c c c c} \hline
**Watermark Detected** & \multicolumn{3}{c}{**ST Edge AI**} & \multicolumn{3}{c}{**Original Framework**} \\ \hline  & **VWW Dataset** & **BER** & **VWW Dataset** & **BER** \\  & **[\%]** & **[\%]** & **[\%]** & **[\%]** \\ \hline \multicolumn{4}{l}{**Original models**} \\ UnWMd MobilenetV1 & 78.25 & 43.75 & 78.25 & 43.75 \\ WMd MobilenetV1 & 71.66 & 0.00 & 71.66 & 0.00 \\ \hline \multicolumn{4}{l}{**Gaussian attacks**} \\
0.001 & 60.93 & 0.00 & 60.93 & 0.00 \\
0.01 & 60.97 & 0.00 & 60.97 & 0.00 \\
0.1 & 58.68 & 0.00 & 58.68 & 0.00 \\
1 & 50.00 & 0.00 & 50.00 & 0.00 \\
10 & 50.00 & 8.98 & 50.0 & 8.98 \\ \hline \multicolumn{4}{l}{**Prune attacks**} \\
0.1 & 59.96 & 0.00 & 59.96 & 0.00 \\
0.2 & 58.11 & 0.00 & 58.11 & 0.00 \\
0.3 & 56.84 & 0.00 & 56.84 & 0.00 \\
0.4 & 53.21 & 0.00 & 53.21 & 0.00 \\
0.5 & 50.68 & 0.00 & 50.68 & 0.00 \\ \hline \multicolumn{4}{l}{**Quantization attacks**} \\
16 bit & 61.01 & 0.00 & 61.01 & 0.00 \\
8 bit & 60.90 & 0.00 & 60.90 & 0.00 \\
7 bit & 59.94 & 0.00 & 59.94 & 0.00 \\
6 bit & 59.79 & 0.00 & 59.79 & 0.00 \\
5 bit & 57.09 & 0.00 & 57.09 & 0.00 \\ \hline \end{tabular}
\end{table}
Table 13: Watermark robustness evaluation on UnWMd and WMd attacked models with white-box method, for the Section 5.1 use case, using ST Edge AI and the original framework. The results for which the origin of the watermark can be recognized are highlighted in green.