Paper: Who Leaked the Model? Tracking IP Infringers in Accountable Federated Learning (ICLR-2024)

Reviewer comment (Novelty):
The discussion of related work primarily mentions FedTracker as relevant prior work. However, there also exist further works such as Merkle-Sign by Li et al. (arXiv:2105.03167 / ICMEW'22) and FedCIP by Liang and Wang (arXiv:2306.01356).

Author response:
Thank you very much for your comment! Both Merkle-Sign and FedCIP are client-side watermarking, which is different from our server-side watermarking. Client-side watermarking is used to claim the co-ownership of the model, yet we argue that client-side watermarking has some limitations, which makes it not applicable for IP tracking. For client-side watermarking, if one of the clients is the infringer to illegally distribute the model, the infringer will not reveal their own identity during the model verification process in order to avoid legal responsibility. Even if the ownership of the model can be claimed by their co-author, the real infringers cannot be tracked, since they remain anonymous. Using our server-side watermark, there is no such concern, the server can easily track the malicious client among all the clients. We add the discussions of these approaches in A. Discussions “Client-side watermarking VS server-side watermarking” in the supplementary.

Reviewer comment (Reproducibility):
The generation of the trigger sets based on the pre-trained encoder of Li et al is not really explained. It would be great to get some more details how the encoding of client keys into the dataset works.

Author response:
The pre-trained encoder has the same architecture as Li et al. During the pre-training of the encoder, we encode an attacker-specified string into benign images through an encoder-decoder network (The decoder here will not be used in our paper). The mapping from the string to the target label will be generated when DNNs are trained on the poisoned dataset. The generated triggers are invisible additive noises containing a representative string of the target label.

Reviewer comment (Presentation):
Algorithm 1, instead of simply referring to Equations 6 and 1, should make it more explicit where some of the defined values such as the set D_T are being used.

Author response:
Thank very much for your suggestion. In the revised version of Algorithm 1, we add how trigger sets, and local training sets are used.

Reviewer comment (Presentation):
Instead of providing only the final benchmark results after all rounds in Table 1, plots showing the evolvement over rounds would be interesting.

Author response:
Thank you very much for your suggestion. The Acc and WSR curve with respect to the communication rounds can be found in Figure 4b. Acc will decrease once the watermark is injected, but will quickly resume after a few rounds. WSR will increase to a high value, and then remain stable.

Reviewer comment (Evaluation):
The method is impractical for FL across devices where we can deal with millions or more clients. It assumes that "During each communication round, the server watermarks the aggregated global model using the client-wise trigger sets before dispatching the model." and it aims at a "traceable IP verification for accountable FL that can accurately identify the infringers among a **scalable** number of clients". It was remarked that the early training rounds can be skipped but only the first 20 out of a total 300 for CIFAR10 (beginning of page 7). Furthermore: "in order to avoid this pitfall, we have to ensure the uniqueness of both the triggers and target labels between different clients". Overall, this method is excessively expensive for the server.

Author response:
Thank you very much for your comment. In this paper, instead of a cross-device setting, we target a cross-silo setting of FL, which is also a widely adopted and practical setting [1-3], including FL IP protection setting [4]. In the cross-silo setting, each client is an institute, like a hospital or a bank. The number of clients is usually set as hundreds.  Our work aims to prototype and focus on an IP tracking technique and leave the scalability for the future. To address the confusion, in the updated version of the paper, we have add the corresponding clarification at the beginning of section 2.

We also add ablation studies with more clients in Table 8 in the supplementary. According to the results, even with 600 clients, the WSR is still over 73%, and the TAcc remains 100%. With more clients participating in FL, we can still track the malicious client correctly with high confidence. Note that the largest number of clients in the table is not the upper limit of the capacity of our proposed DUW. 

[1] Bagdasaryan E, Veit A, Hua Y, et al. How to backdoor federated learning[C]//International conference on artificial intelligence and statistics. PMLR, 2020: 2938-2948.

[2] T Dinh C, Tran N, Nguyen J. Personalized federated learning with moreau envelopes[J]. Advances in Neural Information Processing Systems, 2020.

[3] Zhu Z, Hong J, Zhou J. Data-free knowledge distillation for heterogeneous federated learning[C]//ICML2021.

[4] Tekgul B G A, Xia Y, Marchal S, et al. Waffle: Watermarking in federated learning[C]//2021 40th International Symposium on Reliable Distributed Systems (SRDS). IEEE, 2021.

Reviewer comment (Evaluation):
There can be a false positive if the client has some additional data from the data used for the watermarking and the potential watermark collisions between different clients.

Author response:
Thank you very much for your comment. Our experiment results in Table 5 and Figure 4b show that even if we use random noise or randomly generated jigsaw for watermark injection we can get 100% TAcc, suggesting no collision of clients. If random noise or jigsaw is applied for watermark injection, the clients will not get the data that shares the same distribution as the watermark data.

Reviewer comment (Evaluation):
The watermark is broken at the very core - if we test the ownership by sending the trigger sets produced for each client, then this requires a lot of queries.

Author response:
Thank you very much for your comment. We agree that this may pose an additional challenge to individual watermarks, and we have proposed a solution that combines client watermark and a global watermark, denoted as a Hybrid watermark, which is shown in B.4 in the supplementary. Our DUW can also be combined with existing global unified watermarks. The unified watermark is to detect the mode leakage first in a black-box way. Only if we can claim ownership of the model using the global unified watermark, we will try to track the identity of the infringers by adopting  DUW. Thus, we can avoid sending so many queries to all the suspect models. We will only send client-wise trigger sets to the models that we have already claimed ownership of to find the anonymous infringer.

Reviewer comment (Experiments):
If there are many Sybils or colluding parties, they could use the same encoder from Li et al. (2021b) to embed the watermark. The method would detect the same watermark for many models, which would make the verification of the client that leaked the model impossible since it is not a single client that leaks the shared model.

Author response:
We want to argue that the watermark injection is not conducted on the client side, it is conducted by the server, which ensures a unique watermark is injected for each client before distributing the watermarked model to the clients. Thus, it is not possible to detect the same watermark for different clients’ models.

Reviewer comment (Reproducibility):
The authors did not release the source code so it is not possible to check the details of the method.

Author response:
We have revised the paper to provide more details for reproducibility. We promise to release codes upon acceptance.

Reviewer comment (Writing):
On page 5, Subsection 3.4 $M_s$ is used for both $(f_s , h_s )$ and $(f_s, D)$.

Author response:
The suspect model is originally constructed of $(f_s,h_s)$, and then we reconstruct it as $(f_s,D)$. Thus, we use the same $M_s$.

Reviewer comment (Presentation):
Figure 1 is too complex and difficult to understand here - what is the decoder?

Author response:
We introduce the architecture of the decoder D in step2 in section 3.2. The decoder $D$ only has one linear layer, whose input dimension is the same as the input dimension of classifier head $h$, and its output dimension is the length of the key.  Figure 1 of the revised manuscript is also revised for easier understanding.

Reviewer comment (Reproducibility):
Page 2 - method description - what is the pre-trained encoder?

Author response:
The pre-trained encoder has the same architecture as [1]. During the pre-training of the encoder, we encode an attacker-specified string into benign images through an encoder-decoder network (The decoder here will not be used in our paper). The mapping from the string to the target label will be generated when DNNs are trained on the poisoned dataset. The generated triggers are invisible additive noises containing a representative string of the target label. 

[1]  Y. Li, Y. Li, B. Wu, L. Li, R. He, and S. Lyu. Invisible backdoor attack with sample-specific triggers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages16463–16472, 2021.

Reviewer comment (Writing):
At the end of page 2: "our work can be summarized in four folds" - but you have only 3 contributions enumerated.

Author response:
Thanks for the catch, and we have corrected this error.

Reviewer comment (Writing):
From the initial description on page 2 - it should be already explained how the watermark despite being produced per client by the server affects the aggregation of the model updates/parameters.

Author response:
Thank you very much for your comment. We investigate how the watermarking will affect model aggregation in Figure 4 (b). Compared with standard training without watermarking (purple line), the standard accuracy has a faster recovery of the standard accuracy after the accuracy drop at the watermark injection round. The final standard accuracy degradations of watermarked model are only 2.34\%, 0.03\% and 0.63\% for digits, cifar10, and cifar100, respectively.

Reviewer comment (Writing):
"Distributed learning framework that enables massive and remote clients" page 3 - what are the massive clients?

Author response:
In our cross-silo FL setting, massive clients indicate a large number of participating institutes. The participating institutes (clients) are typically large in number and have slow or unstable internet connections.

Reviewer comment (Experiments):
Page 2 or 3 - I would like to learn how big have to be the separate dataset/trigger sets $D_T$ for each client. How much additional data does the server have to prepare? How much different the datasets have to be for each client?

Author response:
The size of the trigger set is 500 as stated in section 4 datasets. A client-unique key is encoded into the trigger sets using the pre-trained encoder. The difference between the trigger sets can be demonstrated by the zero watermark collision (100\% tracking accuracy) between clients.

Reviewer comment (Evaluation):
"The server will inject a unique watermark for each client" - again, this exerts the whole work on the server - which is too big of an overhead.

Author response:
The additional computation for the server is decided by the number of watermark injection steps T_w. We found that WSR could reach 99\% just within $T_w=10$ steps. The server can embed the watermark parallelly for all the clients. Since the watermarked model for each client is independent and has no sequence relationship with each other, there is no need to serialize it. Not all clients will be active for each communication round, and the client-unique watermark will only be injected to the active clients. Thus, the computation cost will not be so expensive for the server.

Reviewer comment (Theory):
The proposed method is based on the assumption that the client set is known and therefore an ID string can be assigned. In reality client sets are dynamic, especially in cross-device FL settings. How will the proposed algorithm deal with dynamic increase or decrease of the client set? Also since the decoder's dimension is higher than the number of clients, will this create scalability problems when the number of clients grow very large (e.g. millions)?

Author response:
Thank you very much for your comment. In this paper, instead of a cross-device setting, we adopt a cross-silo setting of FL, which is also a widely adopted and practical setting [1-3], including FL IP protection setting [4]. In the cross-silo setting, each client is an institute, like a hospital or a bank. The number of clients is usually set as hundreds. To address the confusion, in the updated version of the paper, we have add the corresponding clarification at the beginning of section 2.

The decrease in the client number will not affect the effectiveness of the method. As for the increase in client number, we add ablation studies with more clients in Table 8 in the supplementary. According to the results, even with 600 clients, the WSR is still over 73\%, and the TAcc remains 100\%. With more clients participating in FL, we can still track the malicious client correctly with high confidence. Note that the largest number of clients in the table is not the upper limit of the capacity of our proposed DUW. We believe that protecting high-performance models is more valuable in IP protection. Yet, scaling the client number to millions with limited data may cause poor performance. Our work aims to prototype and focus on an IP tracking technique and leave the scalability for the future.

Reviewer comment (Experiments):
The experimental results do not compare with other baseline methods. Are there any other backdoor watermarking approaches that worth comparing with?

Author response:
Thank you very much for your comment. Since tracking IP infringers with backdoor-based watermarking remains a new and open problem, and there is no existing research discussing a feasible solution. Therefore, we compare our proposed DUW with baselines designed by ourselves. Due to the reason that if all the clients share the same trigger, watermark collision will definitely happen, we design different triggers for different clients. Specifically, we use traditional backdoor-based watermarking by adding arbitrary badnet triggers using random noise or 0-1 coding trigger for each client. The target label for each client is set as (client\_{ID} \% class\_number). According to the results, traditional backdoor-based watermarks can only achieve a tracking accuracy lower than 13\% (it will even be lower with the increase of the communication rounds), which is much lower than the 100\% tracking accuracy we have achieved. Note that, the rate of clients with watermark collisions can be calculated as 1-TAcc. The results are shown in in section 3.1 in the paper, and Appendix B.1 in the supplementary.

Reviewer comment (Writing):
In Eq 6 and algorithm 1, \theta_k^f appears from nowhere without clear explanations. I suppose it is the feature exactor of \theta_k, is it correct?

Author response:
Thank you very much for your comment. ${\theta_k=(\theta_k^f, \theta_k^h)}$ is the parameter which will be broadcast for client $k$  by the server, where $\theta_k^f$ is the feature extractor, and $\theta_k^h$ is the classifier head. In the revised manuscript, we add this definition before Eq. (4), where $\theta_k^f$ first appears.