Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper studied the problem of privacy-preserving logistic training on encrypted data. The quadratic gradient is proposed to implement logistic regression training in the homomorphic encryption domain. The proposed method can be seen as an extension of the simplified fixed Hessian and an enhancement of the Adaptive Gradient Algorithm (Adagrad). The implemented homomorphic logistic regression training obtains a comparable result by only 3 iterations.
strengths_and_weaknesses: Strengths: 

This paper studies an important problem. Privacy-preserving logistic regression training is one popular method to protect the training data privacy, especially when logistic regression is ubiquitous in the real world.
The paper provides extensive experimental details and implemented codes that are helpful for reproducibility. 

Weakness:

The writing is not clear to distinguish between the proposed methods and previous methods. Most of section 3 are previous works, instead of proposed methods. For example, section 3.1 about the Simplified Fixed Hessian method is actually from previous works. It is better to write it in a background section. Section 3.2 seems to be the proposed method. Thus, it should be highlighted and clarified using diagrams, and clear descriptions. More importantly, it is not clear to know the motivation why the authors propose a quadratic gradient. What are the observations and insights that we have to using quadratic gradient? 

Typos: for example: lines 183-184. Ower-->owner

More comparisons are needed. Although the authors mentioned [1], it is required to have a result comparison with [1] since the techniques are applicable to logistic regression training. It is also better to compare with [2]. 

The results are not robust or generalized. For example, the enhanced method in the iDash dataset of figure 1 is not robust and may be worse than the baseline method. It would be better to explain more about the reason. Otherwise, it may show that the proposed method is not generalized to other datasets. Also, in table 2, the proposed method has worse accuracy than the baseline method on the Edinburgh dataset.  

[1] Han, K., Hong, S., Cheon, J., & Park, D. (2019). Logistic regression on homomorphic encrypted data at scale. In Proceedings of the AAAI conference on artificial intelligence (pp. 9466–9471).

[2] Bergamaschi, F., Halevi, S., Halevi, T., & Hunt, H. (2019). Homomorphic training of 30,000 logistic regression models. In International Conference on Applied Cryptography and Network Security (pp. 592–611).


questions: -
limitations: -
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper presents a faster gradient variant to implement logistic regression training in a homomorphic encryption domain. The results show the new optimization method could speed up the convergence of training on small datasets.
strengths_and_weaknesses: Strength:
1. This paper is well-written and clearly organized, including sufficient theoretic analysis of the proposed Simplified Fixed Hessian method. I like such a solid line of work.
2. The targeted topic – privacy-preserving domain of logistic regression is very promising and deserves insightful hard work to resolve the application of homomorphic encryption.
3. The results of convergence speed are clear and convincing.

Weakness:
1. The experiments are somewhat not enough, considering the lack of experimental support for the claim in line 44 “without compromising much on computation and storage.” It will be better if a detailed comparison of computation and storage is presented.
2. The algorithm may lack of generalization to some large-scale datasets, considering the calculation of $\bar{\math{B}}$ needs to walk through the whole dataset, which is quite resource-hungry
3. Lack of further analysis of the results of the experiment, such as in Figure 1(a), why the convergence speed of Enhanced Adagrad is worse than the raw Adagrad.

questions: 1. In Figure 1(a), why the convergence speed of Enhanced Adagrad is worse than the raw Adagrad?
2. I am curious about the consumption of memory in the HE domain, it will be better if the authors include a comparison between the proposed method and the baseline method.
3. I like the idea of the second-order optimization and wonder whether the SFH method could be applied to more general NNs like CNN, LSTM or Transformer?

limitations: Yes, the authors have included the limitations in line 119, which includes (a) the numerical condition of the SFH method and (b) the singular problem for some high-dimensional sparse matrix datasets.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This study suggests an enhanced gradient method on logistic regression with homomorphic encryption (HE)
The contents of the paper generally follows (Kim et al., 2018), so for who read the paper the contents are familiar.
The others modified existing algorithm by replacing learning rate with quadratic gradient, which is a variant of simplified fixed Hessian.
Experimental results show that the proposed method converges better than existing method in (Kim et al., 2018).
strengths_and_weaknesses: Strengths
- In table 2, It is empirically shown that the proposed method can achieve similar accuracy compared to (Kim et al., 2018) with much less computation time. (the ratio of the computation time is bigger than 7/3 for most datasets, which seems because bigger $logq=60$ was used for the proposed method)
- By transferring the role of computing $\tilde{B}$ to the data owner, the proposed method does not require more multiplicative depth compared to (Kim et al., 2018).
- The paper seems clear and easy to follow.

Weaknesses
- The novelty seems rather shallow; (Kim et al., 2020) has already proposed an approximate Hessian which is very similar to quadratic gradient method. The authors should check the paper.
- Algorithm 1 seems rather meaningless. The algorithm should incorporate operations on ciphertexts, which deal with ciphertext packing.

(Kim et al., 2020)  Kim, M., Lee, J., Ohno-Machado, L., & Jiang, X. (2019). Secure and differentially private logistic regression for horizontally distributed data. IEEE Transactions on Information Forensics and Security, 15, 695-710.
questions: 1. Parameter setting is an important issue when using HE, because parameter search is extremely unfriendly with HE. In algorithm 1, the authors set $\alpha_{t+1}=(1+\sqrt{1+4\alpha_{t}^2})/2$, which is different to the setting in (Kim et al., 2018), which is $\alpha_t=10/(t+1)$. Can the parameter choice be justified?

2. In the experiments, iDash dataset showed different results than other datasets. Can it be explained?

3. Can the results with $\lambda=128$ be given?

4. In table 2, the auc score of Ours on nhanes3 is too low. Is it a typo?
limitations: The authors have not address the limitations of their work. Because the study is about privacy-preserving logistic regression, it seems to have a positive societal impact.

As mentioned above, the biggest limitation of their work is that there is no significant improvement compared to the existing methods.
review_title: null

Meta-review: metareview: Reviewers remained concerned about the novelty of the contribution, about the extent/limitations of experiments/comparisons to other methods, as well as about the fact that the method does not seem to outperform competitors in certain cases.
confidence: Certain
Decision: Reject