# RoSe-Mix: Robust and Secure Deep Neural Network Watermarking in Black-Box Settings via Image Mixup
## 4 Experimental Results
### Robustness  
The watermark must also be robust against various attacks, such as fine-tuning, quantization, and pruning. Since RoSe-Mix operates in a black-box setting, its performance is evaluated using the standard recovery metrics: test accuracy (TA), training set watermark recovery Rate (\(Rec_{T\!\!T}\)), and test set watermark recovery rate (\(Rec_{T\!\!S}\)). The recovery metric is commonly used in black-box watermarking methods, but we extend it to training and testing. Together, the recovery and the test accuracy metrics reflect the watermark retrieval success based on the model's behavior (output to triggers) rather than its internal parameters. Unlike white-box watermarking, where bit error rate (BER) or weight perturbation analysis might be applicable, black-box watermarking relies on behavioral patterns observable from the model's predictions. This ensures that the verification process remains practical and does not require access to the model's internal structure. Additionally, we incorporate dynamic quantization, full unsigned 8-bit quantization, and full signed 8-bit quantization into our robustness evaluation to ensure a comprehensive assessment of \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Metric**} & \multirow{2}{*}{**Host DNN**} & \multicolumn{2}{c}{**Watermarked**} & \multirow{2}{*}{**Fine-Time**} & \multirow{2}{*}{**Dyn. Quant.**} & \multicolumn{2}{c}{**Full Units.**} & \multicolumn{2}{c}{**Full Ints.**} & \multicolumn{2}{c}{**Float16**} \\  & & & & & **Quant.** & & **Quant.** & **Quant.** & **JPEG55** \\ \hline \multicolumn{8}{c}{MNIST} \\ TA & 98.64 & 99.11 & 99.11 & 99.14 & 99.14 & 99.14 & 99.11 & 99.09 \\ \(Rec_{T\!\!T}\) & 13.54 & 100 & 100 & 100 & 100 & 100 & 100 \\ \(Rec_{T\!\!S}\) & 12.50 & 100 & 100 & 100 & 100 & 100 & 100 \\ \hline \multicolumn{8}{c}{FashionMNIST} \\ TA & 88.44 & 88.63 & 88.63 & 88.62 & 88.62 & 88.62 & 88.63 & 87.73 \\ \(Rec_{T\!\!T}\) & 8.33 & 89.4 & 89.4 & 90.2 & 90.2 & 90.2 & 89.4 & 88.6 \\ \(Rec_{T\!\!R}\) & 7.8 & 88 & 88 & 88 & 88 & 88 & 88 & 88.4 \\ \hline \multicolumn{8}{c}{Cifar10} \\ TA & 70.52 & 78 & 78 & 78.04 & 78.04 & 78.04 & 77.99 & 75.42 \\ \(Rec_{T\!\!T}\) & 10.42 & 100 & 100 & 100 & 100 & 100 & 100 \\ \(Rec_{T\!\!T}\) & 7.29 & 96.8 & 96.8 & 96.8 & 96.8 & 96.8 & 96.8 & 96.4 \\ \hline \multicolumn{8}{c}{Transfer Learning} \\ TA & 86.54 & 86.57 & 71.8 & 86.63 & 86.63 & 86.63 & 86.56 & 80.36 \\ \(Rec_{T\!\!T}\) & 11.46 & 100 & 67 & 100 & 100 & 100 & 100 & 100 \\ \(Rec_{T\!\!S}\) & 9.38 & 100 & 70.2 & 100 & 100 & 100 & 100 & 100 \\ \hline \multicolumn{8}{c}{GTSRB} \\ TA & 78.89 & 90.1 & 80.84 & 90.08 & 90.08 & 90.08 & 90.1 & 86.45 \\ \(Rec_{T\!\!T}\) & 2.2 & 100 & 89.4 & 100 & 100 & 100 & 92.8 \\ \(Rec_{T\!\!T}\) & 1.4 & 88 & 94.8 & 88.6 & 88.6 & 88.2 & 79.4 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: RoSe-Mix performance metrics: TA, \(Rec_{T\!\!P}\), and \(Rec_{T\!\!S}\) under various attacks. Rose-Mix against commonly used model compression techniques. We can also highlight that each watermarked model has a better TA compared to its version without a watermark. This effect is often observed in backdoor injection [15], which acts as learning with noisy-labeled samples [40]. We begin our analysis with Table 1, which shows that RoSe-Mix demonstrates high robustness to fine-tuning and quantization attacks, as test accuracy (TA) and watermark recovery rates (\(Rec_{T}\) and \(Rec_{Ts}\)) remain almost unaffected across most datasets. For instance, in MNIST, FashionMNIST, and CIFAR-10, \(Rec_{Ts}\) **remains larger than 88%** across all tested attack scenarios, highlighting that RoSe-Mix preserves the watermark even under significant post-processing. The slight increase in \(Rec_{T}\) for FashionMNIST under quantization (by 0.8%) suggests that the Mixup augmentation process improves the generalization of the model rather than degrading its fidelity or performance of the primary task. The impact of JPEG compression is more significant.