# Watermarking Tiny MLCommons Image Applications Without Extra Deployability Costs
## 9 Attacks and Results
### Results for the Black-Box Watermarking Method  
In this subsection, the resistance to attacks explained above will be presented for WMd and unWMd models with datasets explained in Sections 5.1 and 5.2 considering the watermarking algorithm explained in Section 6.2 embedded using abstract images generated by the Pillow library as described in Section 6.2.5. In addition to the results from the inferences with the original dataset as input, the accuracy value obtained using the selected trigger value for each model has been reported in Tables 13 and 14. To ensure the correct identification of watermarks, the accuracy of the trigger dataset should exceed 88%, with a non-recognition threshold of 12%. Results that met this criterion were highlighted in green. The results of experiments discussed in Section 7.2.1 and reported in Table 15 revealed varying degrees of effectiveness for each attack on the watermarked model that uses the dataset explained in Section 5.1 that uses a MobilenetV1 model with the Sections 6.2 and 6.2.2 approaches composed of random shapes and colors. In the case presented, a similar behavior of the model subjected to the three types of attacks (Gaussian, prune and quantization), obtained in the two previous tables, in terms of accuracy, was also found for Table 15. The attack does not seem to have been decisive from the beginning, but it was possible to notice that as the power increased, the accuracy decreased, reaching a minimum of 50%. The trend of the data is also very consistent with the level of attack applied to the model. According to the results obtained in Table 15, the WMd model appeared to be more robust with the application of quantization attack, obtaining a percentage higher than 88% up to the set 6-bit attack value, while it seemed to be more susceptible to the pruned attack in which, already from the second level of attack, the trigger dataset reached a percentage of 77.46%. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Watermark Detected** & \multicolumn{2}{c}{**ST Edge AI**} & \multicolumn{2}{c}{**Original Framework**} \\ \hline  & **Cifar10** & **BER** & **Cifar10** & **BER** \\  & **[\%]** & **[\%]** & **[\%]** & **[\%]** \\ \hline
**Original models** & & & & \\ UnWMd ResNet-8 & 80.67 & 61.71 & 80.67 & 61.71 \\ WMd ResNet-8 & 75.04 & 0.00 & 75.04 & 0.00 \\ \hline
**Gaussian attacks** & & & & \\
0.001 & 75.00 & 0.00 & 75.00 & 0.00 \\
0.01 & 74.63 & 0.00 & 74.63 & 0.00 \\
0.1 & 73.59 & 0.00 & 73.59 & 0.00 \\
1 & 10.74 & 0.00 & 10.74 & 0.00 \\
10 & 10.03 & 26 & 10.03 & 26 \\ \hline
**Prune attacks** & & & & \\
0.1 & 75.08 & 0.00 & 75.08 & 0.00 \\
0.2 & 74.69 & 0.00 & 74.69 & 0.00 \\
0.3 & 73.96 & 0.00 & 73.96 & 0.00 \\
0.4 & 70.21 & 0.00 & 70.21 & 0.00 \\
0.5 & 61.25 & 0.00 & 61.25 & 0.00 \\ \hline
**Quantization attacks** & & & & \\
16 bit & 75.04 & 0.00 & 75.04 & 0.00 \\
8 bit & 74.73 & 0.00 & 74.73 & 0.00 \\
7 bit & 71.06 & 0.00 & 71.06 & 0.00 \\
6 bit & 56.41 & 0.00 & 56.41 & 0.00 \\
5 bit & 57.61 & 0.00 & 57.61 & 0.00 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Watermark robustness evaluation on UnWMd and WMd attacked models with white-box method, for the Section 5.2 use case, using ST Edge AI and the original framework. The results for which the origin of the watermark can be recognized are highlighted in green. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Watermark Detected** & \multicolumn{3}{c}{**ST Edge AI**} & \multicolumn{3}{c}{**Original Framework**} \\ \hline  & **Cifar10** & **Trigger Dataset** & **Cifar10** & **Trigger Dataset** \\  & **[\%]** & **[\%]** & **[\%]** & **[\%]** \\ \hline \multicolumn{4}{l}{**Original models**} \\ UnWMd ResNet-8 & 80.67 & 38.09 & 80.67 & 38.09 \\ WMd ResNet-8 & 83.44 & 100 & 83.44 & 100 \\ \hline \multicolumn{4}{l}{**Gaussian attacks**} \\  
\hline \hline \end{tabular}
\end{table} 
Table 15: Watermark robustness evaluation on UnWMd and WMd attacked models with black-box method, for the Section 5.1 use case, using ST Edge AI and the original framework.