Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary:
The authors combine several cryptographic techniques to create a federated systems that allows several entities to run classification against all the model held be the participants without revealing information in the process. In particular, the sample to be classified is not revealed to any other party, and differential privacy is used to protect the training data that was used to train the models. A central semi-honest coordinator is used to aggregate the results and add the differential privacy without learning any private information.

Pros:
The strength of this works lie in combining relevant techniques and to show experimentally that the resulting system does improve over using a local model both when the training is distributed evenly or in skewed manner while taking privacy considerations into account.

Cons:
- From a cryptographic point of view, the combination of techniques is somewhat expectable.
- I'm wondering about the low statistical security (${2^-23}$). This seems to be related to the usage of (unbounded) integer secret sharing. Would it be possible to use secret sharing modulo an integer, in which case the security could be perfect?
- I think it would be easier to follow if steps 1-3 were combined in the description because they all take between the same pairs of parties. The exact techniques used don't seem to matter as long as the output secret sharing is the desired result, namely the one-hot vector.
- I find the term collaborative learning somewhat overblown because the proposed protocol only runs classification collaboratively.

Overall:
Despite the points above, I'm in favor of acceptance because the paper seems to improve on previous work, and because it is written very well.

Minor issues:
3.3: leakeage
4.1: odd juxtaposition in the formatting of "arg max" and "sum"
Figure 3: very hard to read in black-and-white

review_title: Solid but unsurprising system for federated classification system with privacy

final_score: rating: 7: Good paper, accept
review_content: 

This work motivated by healthcare and finance where separate parties may wish to collaborate and learn from each other's data but are prevented from doing so due to privacy regulations. This paper propose Confidential and Private Collaborative (CaPC) learning, the first method provably achieving both confidentially and privacy in a collaborative setting. This work also discussed about fairness. I liked this part, since it seems very cool. However, I'm not convinced by the method in this work is better than InstaHide (I could be wrong).




Minor comments
In Section 2.1, this paper should be discussed, since it proposed a way to ``encrypt'' the images/texts. 

InstaHide: instance-hiding schemes for private distributed learning
https://arxiv.org/abs/2010.02772
ICML 2020
Yangsibo Huang, Zhao Song, Kai Li, Sanjeev Arora.


TextHide: Tackling Data Privacy in Language Understanding Tasks
https://arxiv.org/abs/2010.06053
EMNLP 2020
Yangsibo Huang, Zhao Song, Danqi Chen, Kai Li, Sanjeev Arora

In Section B, it lists many theorems/definition about differential privacy. In Section C, it list many backgrounds about sampling. In Section D., it list many definitions on Fairness. I don't quite see the point of having them in appendix, since none of them got mentioned in Appendix E, which is the proof of the main theory result in this paper.

This paper is closely related differential privacy. I think this paper should also be mentioned somewhere.

Privacy-preserving Learning via Deep Net Pruning
https://arxiv.org/abs/2003.01876
Yangsibo Huang, Yushan Su, Sachin Ravi, Zhao Song, Sanjeev Arora, Kai Li.

review_title: Fairness seems very cool, but no convinced by privacy part

final_score: rating: 7: Good paper, accept
review_content: This paper works on the problem of collaborative learning while preserving both confidentiality and privacy of the data points. It combines techniques from secure multi-party computation and differential privacy for the same, and improves on confidential inference and PATE in the process. The new technique is called CaPC. Finally, it states empirical results as evidence for the improved accuracy.

Weakness:
1. The evaluation is done on just two datasets. So, it is a little hard to judge whether the techniques would generalise or not.
2. The writing of the paper itself is not that great because it is difficult to understand the low level details of the experiments.
3. They talk very little about improving on the fairness guarantees.

Strengths:
1. Their techniques enable collaborative learning even in settings where the local architectures of different parties are different.
2. The algorithms they provide improve on fairness.
3. Their empirical results are better than the previously known methods.

Evaluation: I believe the combination of secure multi-party computation and differential privacy is not totally new, but since it yields decent results, I would say that the paper deserves a chance to be accepted.


review_title: CaPC Review. Decision: Accept.

Meta-review: This work describes a system for collaborative learning in which several agents holding data want to improve their models by asking other agents to label their points. The system preserves confidentiality of queries using MPC and also throws in differentially private aggregation of labels (taken from the PATE framework). It provides expriments showing computational feasibility of the system. The techniques use active learning to improve the models.

Overall the ingredients are fairly standard but are put together in a new (to the best of my , admittedly limited, knowledge of this area). This seems like a solid attempt to explore approaches for learning in a federated setting with strong limitations on data sharing.
Decision: Accept (Poster)