Paper:
Outlier detection and novelty detection are two important topics for anomaly detection. Suppose the majority of a dataset are drawn from a certain distribution, outlier detection and novelty detection both aim to detect data samples that do not fit the distribution. Outliers refer to data samples within this dataset, while novelties refer to new samples. In the meantime, backdoor poisoning attacks for machine learning models are achieved through injecting poisoning samples into the training dataset, which could be regarded as "outliers" that are intentionally added by attackers. Differential privacy has been proposed to avoid leaking any individual's information, when aggregated analysis is performed on a given dataset. It is typically achieved by adding random noise, either directly to the input dataset, or to intermediate results of the aggregation mechanism. In this paper, we demonstrate that applying differential privacy could improve the utility of outlier detection and novelty detection, with an extension to detect poisoning samples in backdoor attacks. We first present a theoretical analysis on how differential privacy helps with the detection, and then conduct extensive experiments to validate the effectiveness of differential privacy in improving outlier detection, novelty detection, and backdoor attack detection.

Review:
This paper leverages differential privacy’s stability properties to investigate its use for improved anomaly and backdoor attack detection. Under an assumption (called “uniformly asymptotic empirical risk minimization”), the authors show that difference between the expected loss of a differentially private learning algorithm on an outlier (where the expectation is taken over the randomness of the learning algorithm) and the expected loss of the same algorithm on data from the underlying distribution (expectation taken over data & randomness of the algorithm) is lower bounded by a (possibly/hopefully) non-negative quantity with high probability. The authors then conduct a set of experiments to show that differential privacy improves the performance of outliers, novel examples, and backdoor attack detection. 
Overall, the paper is very well written and easy to read. The paper also tackles an important and timely problem that is relevant to the ICLR community. While there has been some recent work on connecting differential privacy to robustness & attacks, this paper investigates the use of differential private model training as a means to improve novelty detection at inference time. 
A few points that need attention from the authors: 
1. The theory developed is insightful in general but has very little (to no) practical value. For starters, it assumes that differentially private model training is uniformly asymptotic to empirical risk minimization. This is not necessarily true for highly non-convex models trained with SGD. Further, it cannot be verify via experimentations (despite the authors’ attempt to sanity check it using Figure 1). More importantly, the theory developed in Section 3 is not used in any meaningful way in the experiments section — the anomaly detection schemes are agnostic to it. 
2. The authors make no attempt to co-optimize the performance of the model with its ability to be used for better anomaly detection. For instance, the authors choose an l2-clipping-norm C of 1 and do not consider trading off C with the noise variance. 
When the training set contains anomalies, this work can be viewed as “what is the impact of differential privacy” on a training sets with a majority group (training examples from a given distribution) and a minority group (training examples from a different distribution). Under this view, this paper essentially says that “differential privacy leads to disparate impact on model accuracy/loss”. This has been recently investigated in the following NeurIPS19 paper: https://arxiv.org/abs/1905.12101. Thus the contributions of the paper are not substantial.