Paper:
In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples. We leverage the sequential composition theory in DP, to establish a new connection between DP preservation and provable robustness. To address the trade-off among model utility, privacy loss, and robustness, we design an original, differentially private, adversarial objective function, based on the postprocessing property in DP, to tighten the sensitivity of our model. An end-to-end theoretical analysis and thorough evaluations show that our mechanism notably improves the robustness of DP deep neural networks.

Review:
This paper propose an algorithm with DP preservation to train adversarially robust neural networks. To preserve DP, a single-layer linear autoencoder with shared weights is learned to extract features from training data, whose encoder is used to extract private features for the training and inference of a deeper network. To enhance robustness against various attacks, adversarial examples crafted with such attacks are injected into the training set in this algorithm. Guarantees of privacy preservation for training on both clean data and adversarial examples for the autoencoder and the inference network are given. Certified robustness of the smoothed classifier is also given, which depends on the privacy budget of each compositing mechanism. Experimental evaluations of two small (2 and 3 conv layers) networks are given on the MNIST and CIFAR10 dataset, showing improved conventional accuracy on clean samples and adversarial attacks, and certified accuracy than 4 baseline privacy-preserving algorithms.
To my knowledge, this paper provides the most comprehensive analysis so far about privacy preservation in the process of enhancing empirical adversarial robustness, as well as the impact of privacy preservation on the certified robustness of the smoothed classifier. However, the current version is quite difficult to follow for people without DP background, with some settings even conflict with other papers on adversarial robustness, and I do have some doubts about the experimental results. Specifically,
1. What is the role of adversarial examples in the proposed algorithm? Perhaps I missed something, but the authors have not shown how it is related to certified robustness in the paper. The adversarial examples seem to be used only for improving the empirical accuracy against various adversarial attacks. Empirical adversarial robustness is usually down weighted to me when certified robustness is given, therefore an algorithm enhancing only the empirical one does not seem so interesting. 
2. What is even more against my intuition at first glance is that the empirical adversarial accuracy in Figure 4 is lower than the certified accuracy in Figure 6 in some situations. After a while I realized that the conventional accuracy against attacks in Figure 4,5 and the certified accuracy in Figure 6,7 are actually talking about different models, where the first one is the deterministic (or one-random-sample) inference network but the second one is for the smoothed classifier (as referred to in by Cohen et al. 2019, or the expectation of the inference network). This distinction should be addressed, since only one model can be chosen at deployment.
3. The attacks used for evaluating the empirical adversarial robustness are too weak. Only 10 steps are used. More iterative steps, e.g., 1000 or 10000 step PGD I-FGSM, should be provided to reveal the actual robustness of the networks in Figure 4,5.
4. The results of the proposed algorithm in Figure 7 is much better than the state-of-the-art, but I cannot see clearly from this paper how such improvement is achieved. With a ResNet110, [1] and [2] can only achieve around 30% certified accuracy when the maximum l_infty norm is 8/255. However, in Figure 7, the proposed algorithm is able to keep the certified accuracy above 40% at much larger perturbations, with only a 3-conv-layer network. If I am understanding the numbers correctly, could the authors explain clearly where such improvements come from?
Therefore, I tend to reject the paper before my concerns are addressed.
[1] "Filling the Soap Bubbles: Efficient Black-Box Adversarial Certification with Non-Gaussian Smoothing", https://openreview.net/forum?id=Skg8gJBFvr
[2] Salman, Hadi, et al. "Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers." NeurIPS (2019).