# KeyNet: An Asymmetric Key-Style Framework for Watermarking Deep Learning Models
## 5 Experimental Results
### Experiments and Results  
First, we made sure that an accurate private model could not be obtained (and therefore the ownership of a DL model could not be claimed) by using only the predictions of a black-box DL model. To do so, we queried different unmarked DL models with all the signed samples in the owner's WM carrier set, and we used the predictions as input features to train the private model. Table 4 shows the performance of the private models obtained in this way after 250 epochs. It can be seen that the average accuracy of the private model at detecting the signature position inside the WM carrier set is as low as 32.27%. This accuracy was obtained by granting unconditional query access to the black-box model and by using its predictions as input to train the private model. Based on that, we decided to set threshold \(T=0.9\), which is nearly three times greater than the above average accuracy. Therefore, to prove her ownership of a black-box DL model, the owner's private model must detect the signature positions in the WM carrier set with an accuracy greater than or equal to 90%. In the following, we report the results of _KeyNet_ on several experiments that test its fulfillment of the requirements depicted in Table 1. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Data Set** & **Black-Box DL Model** & **Watermark Detection Accuracy \%** \\ \hline \multirow{2}{*}{**CIFAR10**} & ResNet18 & 31.25 \\  & VGG16 & 30.14 \\ \hline \multirow{2}{*}{**FMNIST5**} & CNN & 34.42 \\  & LeNet & 33.26 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Accuracy of the private models at detecting the position of the owner’s signature in the WM carrier set when trained for 250 epochs with the predictions of black-box models. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Data Set** & \begin{tabular}{c} **Owner’s WM** \\ **Carrier Set** \\ \end{tabular} & \
\begin{tabular}{c} **Attacker’s WM** \\ **Carrier Set** \\ \end{tabular} &
\begin{tabular}{c} **Attacker’s Private Model** \\ **Architecture** \\ \end{tabular} \\ \hline CIFAR10 & STL10 & Fashion-MNIST & FC(10,20),FC(20,30), FC(30, 6) \\  & & & (980 learnable parameters) \\ \hline FMNIST5 & MNIST & STL10 & FC(5, 20), FC(20,10), FC(10,6) \\  & & & (360 learnable parameters) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Attacker’s WM carrier sets and private models. The attacker’s WM carrier set and private model differ from the owner’s. **Fidelity. ** Embedding the WM should not decrease the accuracy of the marked model on the original task. As shown in Table 5, the marked model's accuracy is very similar to that of the unmarked model. This is thanks to the joint training, which simultaneously minimizes the loss for the original task and the WM task. Furthermore, _KeyNet_ did not only preserve the accuracy in the original task, but sometimes it even led to improved accuracy. That is not surprising, because the watermarking task added a small amount of noise to the marked model, and this helped reduce overfitting and thus generalize better. _KeyNet_ therefore fulfills the fidelity requirement by reconciling accuracy preservation for the original task and successfully embedding of the WM in the target models. **Reliability and robustness. **_KeyNet_ guarantees a robust DL watermarking and allows legitimate owners to prove their ownership with accuracy greater that the required threshold \(T=90\%\). Table 5 shows that WM detection accuracy was almost 100%, and thus our framework was able to reliably detect the WM. We assess the **robustness** of our framework against three types of attacks: _fine-tuning_[45], _model compression_[46; 47], and _WM overwriting_[13; 48]:  
* _Model fine-tuning._ Fine-tuning involves retraining a DL model with some amount of training data. It may remove or corrupt the WM information from a marked model because it causes the model to converge to another local minimum.