Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: In this submission, the authors address a challenging task, in which the goals are three-fold: 1) preserve the privacy of training data in terms of DP; 2) both provably and practically robust to adversarial examples; and 3) maintain high model utility. The authors demonstrate the advantage of the proposed method both theoretically and experimentally. 

Overall, this is a solid work. However, I have the following concerns. If the authors can clarify them during the rebuttal, I am willing to increase my review score.

1) On page 9, the authors mentioned several existing DP-preserving algorithms with provable robustness. Could the authors clearly state the novelty of the proposed method given these existing methods? Thus it would be clear to access the technique contribution of this submission.

2) How about the scalability of the proposed method when applying to real-world scenarios? In the experiment part, the authors show the results on MNIST and CIFAR-10; however, the scale of these two datasets are kind of small compared to real-world applications. Can the authors experimentally show the scalability of the proposed method by increasing the number of samples?

3) This question will not be counted towards the review: Do the authors try other datasets? MNIST and CIFAR-10 are relatively simple image datasets, how about real images? How about non-image datasets? 

4) This submission is longer than 8 pages, it is OK. But the main results are actually in appendix, and the reviewers are forced to read more than 10 pages. It would be better to adjust the contents and have a balanced organization.

A good thing to mention is: It is great to see that the authors conduct experiment comparison and analysis with a relatively small value of epsilon (i.e., epsilon = 0.2).
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper propose an algorithm with DP preservation to train adversarially robust neural networks. To preserve DP, a single-layer linear autoencoder with shared weights is learned to extract features from training data, whose encoder is used to extract private features for the training and inference of a deeper network. To enhance robustness against various attacks, adversarial examples crafted with such attacks are injected into the training set in this algorithm. Guarantees of privacy preservation for training on both clean data and adversarial examples for the autoencoder and the inference network are given. Certified robustness of the smoothed classifier is also given, which depends on the privacy budget of each compositing mechanism. Experimental evaluations of two small (2 and 3 conv layers) networks are given on the MNIST and CIFAR10 dataset, showing improved conventional accuracy on clean samples and adversarial attacks, and certified accuracy than 4 baseline privacy-preserving algorithms.

To my knowledge, this paper provides the most comprehensive analysis so far about privacy preservation in the process of enhancing empirical adversarial robustness, as well as the impact of privacy preservation on the certified robustness of the smoothed classifier. However, the current version is quite difficult to follow for people without DP background, with some settings even conflict with other papers on adversarial robustness, and I do have some doubts about the experimental results. Specifically,

1. What is the role of adversarial examples in the proposed algorithm? Perhaps I missed something, but the authors have not shown how it is related to certified robustness in the paper. The adversarial examples seem to be used only for improving the empirical accuracy against various adversarial attacks. Empirical adversarial robustness is usually down weighted to me when certified robustness is given, therefore an algorithm enhancing only the empirical one does not seem so interesting. 

2. What is even more against my intuition at first glance is that the empirical adversarial accuracy in Figure 4 is lower than the certified accuracy in Figure 6 in some situations. After a while I realized that the conventional accuracy against attacks in Figure 4,5 and the certified accuracy in Figure 6,7 are actually talking about different models, where the first one is the deterministic (or one-random-sample) inference network but the second one is for the smoothed classifier (as referred to in by Cohen et al. 2019, or the expectation of the inference network). This distinction should be addressed, since only one model can be chosen at deployment.

3. The attacks used for evaluating the empirical adversarial robustness are too weak. Only 10 steps are used. More iterative steps, e.g., 1000 or 10000 step PGD I-FGSM, should be provided to reveal the actual robustness of the networks in Figure 4,5.

4. The results of the proposed algorithm in Figure 7 is much better than the state-of-the-art, but I cannot see clearly from this paper how such improvement is achieved. With a ResNet110, [1] and [2] can only achieve around 30% certified accuracy when the maximum l_infty norm is 8/255. However, in Figure 7, the proposed algorithm is able to keep the certified accuracy above 40% at much larger perturbations, with only a 3-conv-layer network. If I am understanding the numbers correctly, could the authors explain clearly where such improvements come from?

Therefore, I tend to reject the paper before my concerns are addressed.

[1] "Filling the Soap Bubbles: Efficient Black-Box Adversarial Certification with Non-Gaussian Smoothing", https://openreview.net/forum?id=Skg8gJBFvr
[2] Salman, Hadi, et al. "Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers." NeurIPS (2019).
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper focus on providing both differential privacy and adversarial robustness to machine learning models. The authors propose an algorithm called differentially private adversarial learning (DPAL) to achieve such goal. DPAL consists two sub-models: (1) An auto-encoder to extract feature representation; and (2) A classifier takes the embedding of encoder and return the predict logits. The auto encoder uses the reconstruction loss and the classifier uses the similar loss as in adversarial learning. 

The auto-encoder takes both real and adversarial examples as input. In order to guarantee  differential privacy to both sub-models, the authors use Functional Mechanism (Zhang et al., 2012), which perturbs the objective function to guarantee the objective is differentially private. To apply functional mechanism, the authors use the 1st-order polynomial approximation of both objective functions (by Taylor Expansion). Laplace noise is injected to both network input and the output of encoder to ensure the objective functions are differentially private (i.e. for any given weight, the influence of individual record on loss value is DP preserved). The authors further show the noise to guarantee privacy can be convert to PixelDP (Lecuyer et al., 2018) and therefore leads to provable robustness against adversarial examples. 

Overall, this paper studies an interesting setting when privacy guarantee and adversarial robustness are both needed in machine learning model and proposes an algorithm to achieve such goal. The paper is well-organized. However, the paper suffers from several critical questions. I do not support the acceptance unless the following questions are well addressed.

1.	The approximation is necessary to derive the privacy guarantee for the objective function. But the paper does not provide either theoretical analysis on the approximation error or empirical result of the distance between the approximate and real function values.  Another way is to use the approximate loss as the objective function to train the models and see how the model performance changes. 
2. The differentially private objective does not lead to differentially private parameters.  In the proof of your Theorem 1, it is clear that the reconstruction loss in Algorithm 1 is differentially private. However, this does not lead to differentially private parameters. In Zhang et al., 2012, the resulting parameters are differentially private only if finding the minimizer of the perturbed objective does not involve any additional information from the original database (see the proof of their Theorem 1). One way is using data-independent grid search to find parameters with low loss. However, Algorithm 1 uses gradient descent to update the parameters, which needs to access the original database at each update. Though the paper uses a perturbed database, the perturbation is not sufficient to provide the privacy level as claimed in Theorem 1. 

Minor comments:
1.	When is \chi_2 in Algorithm 1 being used? In Algorithm1, I do not see its appearance apart from its definition.
2.	At the end of Introduction, you claim DPAL establishes the first connection between DP preservation and provable robustness. However, in the experiments section, SecureSGD and SecureSGD-AGM are given as baseline algorithms which are DP-preserving algorithms with provable robustness. 

review_title: Official Blind Review #3

Meta-review: The authors propose a framework for relating adversarial robustness, privacy and utility and show how one can train models to simultaneously attain these properties. The paper also makes interesting connections between the DP literature and the robustness literature thereby porting over composition theorems to this new setting.

The paper makes very interesting contributions, but a few key points require some improvement:
1) The initial version of the paper relied on an approximation of the objective function in order to obtain DP guarantees. While the authors clarified how the approximation impacts model performance in the rebuttal and revision, the reviewers still had concerns about the utility-privacy-robustness tradeoff achieved by the algorithm.

2) The presentation of the paper seems tailored to audiences familiar with DP and is not easy for a broader audience to follow.

Despite this limitations, the paper does make significant novel contributions on an improtant problem (simultaneously achieveing privacy, robustness and utility) and could be of interest. 

Overall, I consider this paper borderline and vote for rejection, but strongly encourage the authors to improve the paper wrt the above concerns and resubmit to a future venue.
Decision: Reject