Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary: 
This paper studies an important problem and proposes the novel residual perturbation to protect privacy while maintaining the ResNet models’ utility.  Two SDE models are provided to inject noises with abundant theoretical proof are provided. Experimental results demonstrate the performance of privacy protection and classification accuracy on benchmark datasets. My major concern is about the utility enhancement and the DP guarantee (see cons below). Hope the authors can address my concern in the rebuttal period.

Pros: 
1. The paper studies a fundamental problem on how to protect privacy while retaining model utility. The problem itself will have great impacts on real-world scenarios. 
2. The proposed two strategies are novel for injecting noise to each residual mapping of ResNet theoretically principled by the stochastic differential equation theory. Great amounts of theoretical proof are provided and seem technically sound.
3. Experiments on the real-world dataset provide some interesting insights about the advantages of residual perturbation.

Cons: 
1. Experiments demonstrate that utility is enhanced after noise injection. In general, there is a tradeoff between privacy and utility. This paper can increase both, and the authors owe this enhancement to the ensemble of noise injected ResNets. It seems not very clear to me that how to conduct the ensemble. What’s more, if it is just because of the ensemble, what is the contribution of this paper?
2. There should be proof that each iteration of the whole model by strategy I and II can satisfy DP, while authors only prove the parameters can satisfy DP.

Questions during the rebuttal period: Please address and clarify the cons above.
review_title: Interesting topic but there remains some concerns on the utility enhancement the DP guarantee

final_score: rating: 6: Marginally above acceptance threshold
review_content: ### Summary

The paper presents a method for training ResNets with differential privacy. Rather than the usual methods based on noisy gradient descent, the authors propose adding noise at each layer of the network during both training and testing. The authors prove differential privacy guarantees for two strategies of this type (one with additive and one with multiplicative noise). They also show some evidence that the noise can help generalization, by showing that the Rademacher complexity of a continuous linearized version of the model is lower when noise is added.

### Evaluation: Theory

The theoretical results are a start, but have some limitations which seem significant to me:

* Theorem 1 needs the output of any residual mapping to be bounded in expectation. It is not clear whether this actually holds.

* Theorem 2 only gives privacy guarantees for a single prediction. This is interesting but not as a strong as outputting the model.

* Theorem 3 is about a continuous linearized analogue of the models. It also does not have quantitative bounds on how much the Rademacher complexity can improve.

### Evaluation: Experiments

The experiments are promising: the authors run a membership inference attack against their models and against a model trained with DPSGD, to test privacy. They also test accuracy. I am, however, bothered that there is no attempt to give tight differential privacy bounds and to compare the two algorithms with choices of parameters that give the same provable differential privacy bounds. The membership inference attack is supposed to be a proxy for that but I am not convinced that just running one attack and getting a slightly lower AUC is good evidence that the proposed model preserves privacy as strongly as DPSGD. In general, I do not want to see work on privacy in ML adopt the strategy of running a single membership inference attack to verify privacy. This is not convincing: what if a slightly different attack does a lot better? The point of rigorous privacy guarantees is that they hold against all attacks. 

 
review_title: Interesting ideas but results seem preliminary.

final_score: rating: 4: Ok but not good enough - rejection
review_content: The paper at question tackles the well-known problem of differentially private (DP) deep learning:  already for moderate privacy guarantees, the model performance suffers greatly.

The paper proposes a particular SDE based  method for obtaining privacy for ResNets. DP and stochastic differential equations have been considered in conjunction before e.g. in

Wang, Y.X., Fienberg, S. and Smola, A., 2015, June. Privacy for free: Posterior sampling and stochastic gradient monte carlo. In International Conference on Machine Learning (pp. 2493-2502).

Li, B., Chen, C., Liu, H. and Carin, L., 2019, April. On connecting stochastic gradient MCMC and differential privacy. In The 22nd International Conference on Artificial Intelligence and Statistics (pp. 557-566). PMLR.

(you might consider adding these references).

Although providing an interesting approach by combining DP deep learning and SDEs, I think the paper has some major deficits.
One of them is certain sloppiness with the presentation. I do not understand how the forward and backward Euler discretisations correspond to backward and forward propagation of ResNet layers. The main results are regarding the DP-privacy guarantees of the method (Thm. 1 and Thm. 2).
However these DP-guarantees are not used anywhere in the experiments, and it remains a mystery to me what is actually DP protected.
Another criticism is regarding the experiments: when comparing to DP-SGD, there are no eps,delta-values given, it remains mystery how private the method is and how private is DP-SGD for the choice of hyperparameters listed.

The paper would require a major revision and therefore I cannot recommend it for publication in ICLR.

EDIT: The authors have answered my questions and it clarified a lot. Thus I raise my score by one. However, I am still suspicious about the value of the DP result: for example, it is not discussed why would the residual mapping have an L2-sensitivity G (as stated in the assumptions of the theorem). Also, the reported privacy values in the end of the revised version of the paper (epsilon > 1000) are not meaningful. As far as I see, the experiments give an example where this given membership inference attack works better for DP-SGD protected model than for this residual perturbed version. However the paper does not give any privacy guarantees for the residual perturbation method. Whether the L2-sensitivity can be obtained with e.g. batch normalisation remains unclear to me. I think that the paper would require a careful rewriting.

review_title: Deep Learning with Differential Privacy for ResNets Using SDEs

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper focuses on the topic of differentially private deep learning. Specifically, based on the deep residual learning, they first see it as an ODE. Then, to reduce the reversibility of the ODE, they modify the model as an SDE. By discretizing the SDE, they get a perturbed version of residual learning and use this to design DP-algorithms. The first strategy is directly followed the SDE while the second strategy is with an addition multiplicative noise of the additive noise. Finally, they show that their methods to defend membership inference attack both theoretically and practically. 
I tend to accept the paper since I think the paper is well-motivated, also there are privacy guarantees and some theoretical results on defending privacy attack (Theorem 3). However, I still have the following concerns: 

1) It is notable that compared with Strategy 1(S1), S2 can only preserve the prediction privacy (since there is and addition multiplicative noise of the additive noise). So in my opinion, comparing the Rademacher complexity between S1 and S2 is unnecessary. So I want to see more comments about this. 
2) Moreover, the motivation of S1 is clear which is just followed by the SDE. However, the motivation of S1 is unclear, why the authors add an addition multiplicative noise to the additive noise? Is there any other previous work on it? 

review_title: Reivew

Meta-review: This paper proposes techniques for differentially private training of ResNets inspired by SDEs. The idea has some promise but the paper does not give convincing evidence, either theoretical or empirical that it outperforms existing tehniques. Unfortunately, comparisons with existing techniques are presented in a misleading way that does not clearly provide the privacy parameters. 
Another issue with the work is that the authors appear to be unaware of (and hence do not compare with) existing work on privacy of a single prediction (referred to as strategy II in this work). 
Approaches for this problem are described in these theoretical works (and several follow ups)

* Dwork, Feldman. Privacy-preserving Prediction. COLT 2018
* Bassily,Thakkar,Tahkurta. Model-Agnostic Private Learning via Stability. NIPS 2018

Practical results are mentioned in PATE papers of Papernot et al. and more recent work 
* van der Maaten,  Hannun. The Trade-Offs of Private Prediction https://arxiv.org/abs/2007.05089
Decision: Reject