Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes using a cosine activation function and Hadamard-Diagonal transformation as a means to improve the efficiency of MPC for machine learning. The paper considers a two-server model, where the training computation are carried out by two non-colluding parties. Experiments are provided to demonstrate the improvement on the test accuracy of the proposed approach over other baselines that use different activation functions.  
main_review: Strengths- The paper demonstrates the utility of cosine activation function and Hadamard transform in the privacy-preserving machine learning paradigm. The experimental results demonstrate the improvement on the prediction performance over benchmarks is promising. 

Weaknesses-
1- The proposed approach combines well-known building blocks (e.g., cosine activation) that have been studied in the context of ML. Similarly, the secure computing component uses well-known cryptographic primitives such as Beaver's multiplication triplets. 

2 - The proposed setup is for a (non-colluding) two-server model, where the two parties do not cooperate with each other. The two server model is restrictive in practical settings

3- The experiments do not include runtime comparisons between the baselines, or comparing the communication and computation costs, to demonstrate efficiency. 

4- The baselines do not provide an extensive comparison with the former two-party MPC approaches (for instance SecureML [Mohassel-Zhang] also discussed in the related works section). Thus, it would be good to clarify the motivation on the choice of baselines. 



summary_of_the_review: Improving the efficiency and predictive performance of MPC-based techniques for machine learning is an important problem. It would be good to include experiments that demonstrate the efficiency over the benchmarks.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes changes to improve the performance of neural network training with multi-party computation, that is an activation function based on cosine and a linear functionality that reduces the number of weights to be trained.

main_review: While the ideas are interesting, the MPC aspects are insufficient:
- The paper does not name a secret sharing domain. Usually, numbers are shared modulo an integer in order to achieve security via uniform distribution of shares. This modular operation would hinder the summand-wise cosine and sine computations in Algorithm 1.
- It is common to use quantization with MPC because full floating-point computation is much more expensive. The authors don't specify how they have obtained their accuracy results. It would be insufficient to use floating-point computation because it is known that quantization reduces the accuracy [1].

The authors say that it is not efficient to implement ReLU accurately. However, there are implementations that do that, for example [2].

Minor:
- p4: (The) Kernel method is a type of powerful nonlinear
- p6: is a diagonal matrices (matrix)
- p6: Another example is (the) binary embedding

[1] https://arxiv.org/abs/1502.02551
[2] https://arxiv.org/abs/2104.10949


summary_of_the_review: The MPC aspects need more work.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper proposes using a 2-party MPC protocol for evaluating neural networks on private data by replacing activation functions with a cosine function, and another technique of replacing fully connected layers with a specially structured matrix.
main_review: It seems to me that there is a fundamental flaw in how secret sharing is used: computing trigonometric functions on arithmetic secret shares makes no sense, as the shares are over a finite ring. I don't see a way to fix this problem.
summary_of_the_review: Unfortunately, it seems to me that this paper has a fundamental mistake and cannot be accepted.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: MPC is a cryptographic technique to allow multiple party to jointly compute a protocol without leaking sensitive data, but building blocks in the neural network converted to MPC setup usually suffer from heavy communication overhead among parties, and jointly training ML models is also computationally expensive. So, in this work, authors propose an efficient MPC-based neural network. The network consists of cosine function as activation function under 2PC setup and linear transformations by Hadamard-Diagonal method.
main_review: Pros:
1. Compare to ReLU in MPC with multiple rounds of communication, proposed cosine function in 2PC setup only involves two rounds of communication. Since communication overhead is a big deal in MPC among parties, cosine approach decreases the cost.
2. Structure matrices used in the work can reduce bandwidth and save memory.
3. Theoretical explanation for both methods is provided to convince reader in some way. 

Cons:
1. Since this work focuses on MPC approach, it is necessary to state what type of adversary model (semi-honest or malicious) is trying to deal with. More security analysis is expected if possible.
2. Although communication cost is analyzed in words, fail to provide table or charts to show efficiency compared with existing methods, since “efficient” is in the title. With tables or charts would convince more.
summary_of_the_review: To be honest, this paper is below acceptance standard because of innovation and lack of security analysis, although the writing of manuscript looks great. So, currently, I think I would not accept this paper and looking forward to see other reviewers’ comments to make final decision.
review_title: null

Meta-review: There appears to be to be a fundamental error in the paper, w.r.t. the application of the proposed approach to finite fields. As a result, the paper cannot be accepted in its current form.
Decision: Reject