Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The authors propose a method that allows training of UV methods without sharing any user (exemplar or class) embeddings with the server or other uses. Models are trained using gradient averaging on the server, so any leakage through that is not addressed in this work. The paper shows experimental results on speaker identification, face and handwriting verification tasks. The authors argue that this is the first work that considers secure training in a federated setup, with neither raw inputs nor exemplar or class embeddings being shared with the server or other users.

#### Pros

* The paper is clearly written and the derivations are sound (for the most part, see questions below). 
* The idea appears to be novel and a significant delta compared to the SoTa in terms of security and the novelty of a secure embedding learning protocol in the federated setup were only (one) positive classes are available for training.
* The experimental results are promising albeit can't compete with existing less secure methods. 

#### Cons

- Clarity of experiments
  - Especially for the face verification task the code length seems to play a major role. Any discussion giving an understanding of this would be appreciated. Specifically, how and why does $d_{min}$ affect the accuracy. Bottom of page 5 mentions that increasing the code-words and presumably $d_{min}$ increases the performance, but no reasoning is provided.
  - Additional insights of how the baselines (softmax, FedAws) were trained and what the emedding sizes are would be helpful. Is the embedding size ~64 in all cases?

#### Questions & Comments

- The assumption of $||z|| = \sqrt{c}$ should be put into context. What are the practical applications for this assumption. Is it merely there for the math to work out?
- The theorems show that $l_{neg}$ is redundant for when $l_{pos}=0$, however, it is not clear to me that minimizing $l_{pos}$ also corresponds to minimizing $l_{neg}$. In practice, $l_{pos}$ will likely never reach $0$ and a negative loss term could have a significant contribution to the loss surface.
- Page 6 mentions that increasing $l_r$ reduces the minimum distance of the code for a given code length. Why is this the case? Is it because $r_u$ is sampled by the clients and no guarantees can be made? A more detailed discussion would be helpful.

This work proposes a new idea that allows training embeddings for verification with only positive classes in a federated setting, while ensuring security. Some areas could be clarified in the paper, especially why it is sufficient to proof the redundancy of the negative loss term only for the global minimum of when $l_{pos}=0$. Assuming the authors can provide a satisfying explanation, I recommend accepting this work.
review_title: Secure federated learning user verification model training

final_score: rating: 2: Strong rejection
review_content: The paper leverages federated learning to train user verification models. The authors claim that their new federated learning addresses the security and privacy issues of previous methods. In particular, for privacy, the users do not need to send their class embedding vectors to server nor other users. For security, the paper claims that the proposed method is secure against poisoning attacks and evasion attacks. 

Strengths

I think the major strength of the paper is to design a loss function and a way of modeling embedding vectors for users such that the embedding models can be learnt without sharing the embedded vectors to the server nor other users. 

Weaknesses

The paper is weak on its security and privacy claims. 

1. For privacy, can you quantify the privacy leakage of sharing embedded vectors with the server? Without a formal quantification, it is hard to claim your method is more private. 

2. Poisoning attack. I don't think the paper addresses the poisoning attacks. The paper considers that the server may poison the learnt model. However, in the proposed method, the server can still poison the model. In particular, the server can send arbitrary new model to each user. In general, it is hard to defend against malicious server who performs poison attacks. 

Also, malicious users can poison the model training, which are more realistic poisoning attacks. But such poisoning attacks are not considered. I don't see how the proposed method can address these poisoning attacks.  Some references on poisoning attacks:

https://arxiv.org/abs/1807.00459

https://arxiv.org/abs/1911.11815

https://openreview.net/forum?id=rkgyS0VFvr

3. Evasion attack. The proposed cannot address evasion attack at all. 

4. Experimetal details. Can you add more details on experimental details, e.g., learning rate. How is experiment on softmax loss function implemented. 

5. Can you also report AUC to compare different methods, since you already show the true positive rate vs. false positive rate curves?
review_title: unclear security and privacy guarantees

final_score: rating: 6: Marginally above acceptance threshold
review_content: **Summary**
Federated learning takes advantage of the fact that private user data does not need to be transferred and shared across devices or servers. This makes FL particularly attractive for the user verification scenario, where privacy-sensitive biometric data are used to train verification models. One crucial hurdle in this scenario is that per device, only positive data are present, potentially turning the device-wise training objective ill-posed (all embedding are likely to collapse to a single point). As a way to introduce negative examples, FEDAWS has been developed and presented at ICML 2020. This paper recognizes a crucial security risk in the FEDAWS system, that embeddings of user data are transferred to the server, and proposes a more secure training methodology, FEDUV, that involves the error-correcting codes. FEDUV enjoys stronger security guarantees while showing comparable ROC curves as FEDAWS at nearly identical computational costs (though not entirely sure about the computational cost bit ;) ).

**Pros**
The motivation is spot on. Having to see any form of negative samples is the itchy point of the FL-based user verification system. FEDUV magically solves this issue by pre-defining a unique prototype vector for each user, which are not shared across users and are by design far apart from each other (this is the crucial trick!) by employing a technique in error-correcting codes (ECC). As a result, each user's endeavour to get closer to the own prototype vector ensures the maximisation of distance from the others' prototype vectors. 

Three experiments that are quite close to real-world scenarios (speaker, face, and handwriting-based verification) show that the performance of FEDUV is comparable to FEDAWS, the state of the art framework from ICML 2020 with weaker security guarantees.

Writing is nearly flawless. Highly enjoyable paper.

**Cons**
No major cons. Perhaps explain in a bit more depth on the BCH code to illustrate (at least a high-level, hand-wavy description) how it assigns the codes in a distance-maximizing manner. Section 2.3 only explains the desiderata for BCH, rather than *how* BCH achieves it. Please also confirm that FEDUV spends nearly identical computational cost as FEDAWS. Somehow I got this from the paper, but have not found a solid reference that confirms this (if not, please explain, too).

Nits: Please add grid lines and row titles (training set, test set with known users, test set with unknown users) in Figure 2 plots. Baslines --> Baselines. Flatten the last part of Section 1 as paragraphs rather than itemize? Yu et al. 2020 (FEDAWS) is an ICML paper, not arXiv - please fix the reference.

**Key reasons for the rating**
I don't find any major rationale to reject this paper. However, its novelty is also eclipsed by the Yu et al. 2020 (FEDAWS) paper. Though I really like this paper, I believe the best scores should be reserved for more innovative papers.

**After rebuttal & discussion**
I still tend to think that the paper's scope can be adjusted relatively easily (it is not too difficult to insert more disclaimers and change the title), and we can force apply the adjustment by conferring a conditional acceptance.

But I'm sold on the point that there is a lack of argumentation on whether undisclosing the user-specific embedding will improve the privacy guarantee. I had taken this argument as granted, but this is indeed not so obvious, given that there exist many attacks that are applicable in this kind of scenario, as R4 has argued. It would be great if the authors could quantify the improved privacy guarantee.

I'm okay with rejecting the paper then. I still like the paper quite a lot, but rejecting it will also give the authors a good chance to assimilate more points of views in the paper.
review_title: Nearly flawless paper

final_score: rating: 7: Good paper, accept
review_content: In this paper, the authors focus on designing a federated user verification solution. Specifically, the authors address two fundamental challenges associated with user verification, i.e., one-class data (positive data only), and privacy protection (i.e., the raw data and the embeddings of the users and class). Technically, the authors extend a very recent work called FedAWS by (Yu et al., 2020), and introduce a user-specific codewords, which not only protect users' privacy (i.e., not sharing the embedding with other users or the server) but also do not need the negative samples (i.e., the two loss functions in Eq.(5) reduces to one due to equivalence shown in Theorem 1). We can see that the main idea of re-writing the Eq.(2) into two loss functions in Eq.(4) and Eq.(5), and introducing codewords are novel and effective, which also address the two challenges well.

Empirical studies on three user verification cases show the effectiveness of the proposed solution FedUV.

Overall, the technique is novel (and I like this idea) and the paper is well presented. I recommend acceptance.

review_title: Secure Federated Learning of User Verification Models

Meta-review: In this paper, the authors propose to adapt the recent paper by Yu et al. (ICML 2020), namely FedAwS. In that paper, the authors solved a potential failure mode in federated learning, when all the users only have access to one class in their devices. In this paper, the authors extend FedAwS to a setting in which federated learning is used for User Verification (UV), namely FedUV. The authors argue that the previous paper could not be the solution to learning UV because FedAwS share the embedding vectors with the server.
 
The authors then show a procedure in which they can learn a classifier in which the embedding vectors are not needed to be shared with the classifier. They use error-correcting codes to make the mapping sufficiently different and that allows the training to succeed without sharing the embedding. The proposed change is only marginally worse than FedAwS and centralized learning. This is the part of the paper that has attracted positive comments and is praised by all the reviewers.
 
The authors take as given that by not sharing the embedding vectors and by using randomly generated error-correcting codes, the whole procedure is privacy-preserving and secure. The 4th reviewer indicates that these guarantees need to be proven and points out several references that hint toward flaws in the argument by the authors. Reviewer 4th does say that not sharing the embeddings might not be enough, but that self-evident arguments are not enough.
 
This paper provides a significant improvement for a federated machine learning algorithm that deserves publication, but the rationale of the paper is flawed from a privacy and security viewpoint. I think if the paper is published as is, especially with the proposed title, it will create a negative reaction by the security and privacy community for not adhering to their standards. We cannot lower those standards. 
 
I suggest to the authors that they can follow two potential paths for publishing this work:
 
1 Change the scope of their algorithm. For example, I can imagine that by not sharing the embedding the communication load with the server might be significantly reduced or that adding new users with new classes can be easier.
 
2 Follow the recommendation from Reviewer 4 and show that the proposed method is robust against the different attacks.
 
Minor comments:
 
For a paper that is trying to solve the AU problem, I would expect a discussion about why learning is better than a private algorithm. In a way, learning is sharing, and that increases the risk of mischief by malicious users.   
 
The discussion about error-correcting codes and the minimum distance is quite old fashion. In high dimensions, the minimum distance is not the whole story. LDPC codes make sense when we stop focusing on minimum distance codes and minimum distance decoding. I would recommend having a look at the Berlekamp’s Bat discussion in David MacKay’s book (Chapter 13).
Decision: Reject