Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The authors proposed crypten, a new deep learning framework for secure multi-party computation (MPC) based on the PyTorch API. One of the important obstacle in adoption of MPC for deep learning is that using the existing MPC libraries requires extensive knowledge about MPC. A big advantage of crypten is that it follows the same familiar API of PyTorch and hides the MPC complexity behind the API. The provided code snippets demonstrate that using crypten is certainly easy for non-experts. They also support GPU acceleration for arithmetic computation.

main_review: Originality:
- The authors use existing well-known MPC techniques and optimization to create the framework.
- Following the exact API of a well-known deep learning framework, like PyTorch, for a MPC library is novel.
- The authors could've done a better job in reviewing and comparing the existing MPC frameworks for deep learning, for example, a table comparing the features of different frameworks. 


Quality:
- The code snippets and the experiments on different application sufficiently demonstrated crypten's ease-of-use.
- The shortcomings of the framework are also presented and explained.  


Clarity:
- The paper is well written and organized and it is easy to follow.


Significance: 
- The researchers and practitioners will find the work interesting and can probably build on top of the framework.
- The paper is the first to present a framework which follows a well-known deep learning API.

limitations_and_societal_impact: The authors discussed the limitation and shortcoming of the paper and their plan on how to extend the work. They also mentioned the possible broader impact of their work on the society and the community.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a general software framework with GPU support for secure machine learning over multiple (arbitrary number of) parties: CrypTen. It provides a friendly way for people who don't have any cryptography background to implement the secure machine learning tasks, which could tolerate all-but-one corruptions in semi-honest case. It also demonstrates its capability and efficiency in realizing the state-of-the-art models for text classification, speech recognition, and image classification.
main_review: This paper is clearly written and technically sound. The experiments well demonstrates the performance of this software framework. 

Strength: the software framework is easy to use and requires no knowledge of cryptography. It's the first public framework to realize secure modern machine learning tasks among arbitrary number of parties. And it shows a competitive performance for those tasks.

Weakness: 
 - As stated in the paper, it only provides secure against semi-honest adversaries. While other works presents malicious security, but for 2 to 4 parties. 
 - In the current implementation, the security relies on a Trusted Third Party.
 - Most of the techniques were already presented here: https://eprint.iacr.org/2019/599.pdf   I think the authors should also cite this paper.
limitations_and_societal_impact: Those are properly addressed in the paper.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This work is to fill the gaps of software framework for MPC used in Machine Learning. It provides many MPC primitives and fits them into ML scenario.
main_review: Pros:
1. A great contribution to allow ML researcher, who may be lack of solid cryptography knowledge, have a chance to work on MPC-based privacy of ML with relatively easy access to implementation.
2. This work does more than basic ML by providing tensor computation, automatic differentiation and modular neural network to facilitate works deep learning with PyTorch-friendly APIs for state-of-the-art CV and NLP works.
3. With ability to support CrypTen on GPU, it achieves great performance on multiple benchmarks, such as image recognition and speech recognition.

Cons:
1. Although CrypTen did great jobs on passive security allowing an arbitrary number of parties, active security is also crucial to protect ML privacy, and it may helps more on real-world applications, since malicious adversary deviate cryptographic protocols with any arbitrary action.
limitations_and_societal_impact: No potential negative societal impact has seen.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors present an implementation for secure computation of deep learning inference and training. It is based on PyTorch, which makes its use easy for programmers familiar with the underlying framework. The secure computation is semi-honest and uses a trusted third party, which is assumed not to collaborate with any of the other parties.

main_review: I think the effort towards making secure computation more accessible deserves attention, and I appreciate the work the authors have done in this direction.

The authors hint at MPC compilers being potentially faster. What order of magnitude would that be? The authors refer to numerous works, but don't mention any results. For example, CrypTFlow takes 25.9 seconds and 6.9 GB for ResNet-50. In particular, the communication figure seems to be much better than the about 50 GB for ResNet-18 shown in this work.

The paper says that using a ripple-carry adder takes O(log(L)) rounds for L being the number of bits, which is wrong. Ripple-carry addition refers to the textbook addition that O(L) rounds. An adder taking O(log(L)) rounds would be a carry-lookahead adder.

I'm missing an argument in Section 5.3 for how much time using the GPU saves. What is the improvement per clock cycle or second?

The authors say that the results for private training are very similar to inference. I would expect that, in the context MPC, training fares better due to implicit parallelization. Is that what the authors mean here?

I consider the frequent announcement of future features ("we plan to add/support...") out of place in an academic paper, and I would prefer a more neutral style, say "a possible extension would be...".

Minor:
l244: Please move the URL to a footnote or the appendix.
l317: should be Newton-Raphson

limitations_and_societal_impact: No concerns
review_title: null

Meta-review: There is an agreement in the reviews that the paper’s secure multi-party computation (SMPC) framework for deep learning training and inference provides a valuable contribution. While most of the techniques are already known and the protection is only against honest-but-curious (as opposed to malicious) adversaries, the paper seems to fill an important gap and the contributions are a step forward towards broader deployment of SMPC protocols for ML primitives, and could stimulate future work in the area. I therefore lean towards acceptance.
Decision: Accept (Poster)