Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper leverages differential privacy’s stability properties to investigate its use for improved anomaly and backdoor attack detection. Under an assumption (called “uniformly asymptotic empirical risk minimization”), the authors show that difference between the expected loss of a differentially private learning algorithm on an outlier (where the expectation is taken over the randomness of the learning algorithm) and the expected loss of the same algorithm on data from the underlying distribution (expectation taken over data & randomness of the algorithm) is lower bounded by a (possibly/hopefully) non-negative quantity with high probability. The authors then conduct a set of experiments to show that differential privacy improves the performance of outliers, novel examples, and backdoor attack detection. 

Overall, the paper is very well written and easy to read. The paper also tackles an important and timely problem that is relevant to the ICLR community. While there has been some recent work on connecting differential privacy to robustness & attacks, this paper investigates the use of differential private model training as a means to improve novelty detection at inference time. 

A few points that need attention from the authors: 

1. The theory developed is insightful in general but has very little (to no) practical value. For starters, it assumes that differentially private model training is uniformly asymptotic to empirical risk minimization. This is not necessarily true for highly non-convex models trained with SGD. Further, it cannot be verify via experimentations (despite the authors’ attempt to sanity check it using Figure 1). More importantly, the theory developed in Section 3 is not used in any meaningful way in the experiments section — the anomaly detection schemes are agnostic to it. 
2. The authors make no attempt to co-optimize the performance of the model with its ability to be used for better anomaly detection. For instance, the authors choose an l2-clipping-norm C of 1 and do not consider trading off C with the noise variance. 

When the training set contains anomalies, this work can be viewed as “what is the impact of differential privacy” on a training sets with a majority group (training examples from a given distribution) and a minority group (training examples from a different distribution). Under this view, this paper essentially says that “differential privacy leads to disparate impact on model accuracy/loss”. This has been recently investigated in the following NeurIPS19 paper: https://arxiv.org/abs/1905.12101. Thus the contributions of the paper are not substantial. 

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper proposes the idea of using differential privacy (DP) to improve the performance of outlier and novelty detection. Differential privacy was proposed as a privacy metric which limits the contribution of a single data point in the training set to the output. This property naturally controls how poisoned data would affect the output of the learned model. Under the assumption that a well-trained model would incur a higher loss on the outliers, the paper gives a theoretic bound on how this loss will decrease if there are poisoned samples in the training set. 

The paper also performs several experiments on synthetic and real-world datasets. The paper shows that add differential privacy during training can improve the performance of autoencoder-based outlier detection on MNIST data.  For real-world data, the paper improves the performance of anomaly detection on the HDFS dataset over the state-of-the-art algorithm. The paper also shows empirically how DP can help improve backdoor attack detection. 

The paper is overall nicely written with some nice results. The paper could be improved if the following confusions can be resolved.

1. Novelty detection is generally referred to as detecting samples in the test set that are not in the distribution of the training set. In the theory part, the analysis is mostly based on data poisoning, which is not typical in the novelty detection setting. I hope this can be clarified.
2. In the experiment part, the paper uses Figure 1 to show how UAERM is satisfied. I find this a bit confusing. In definition 4, the h^* is referred to as the global minimizer while in the experiment, the empirical minimizer is used.
3. Theorem 2 presents some theoretical bound to show the power of DP on improving outlier detection, however, in the parameter setting used in the experiment, Theorem 2 does not provide meaningful bounds. There is a bit disconnection between the two parts.

Based on the above comments, I think the paper can be accepted if there is room for it. But I won't push it for acceptance.
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: Interesting topic but lacks of novelty
#Summary:
The paper proposes that by applying differential privacy, the performance on outlier and novelty detection can be improved. It first presents a theoretical analysis, which establishes a lower bound on the prediction performance difference between normal and outlier data. By adding noise into the training process, the outliers in the dataset will be hidden by the noise, which will result in a model that utilizes the normal data. In this way, when deploying the model, the model will find the outlier by observing low confidence.

#Strength
It is good to see that the paper builds a connection between the privacy parameter and the noise level and the experiments make the theory valid.

#Weakness
I’m not an expert in differential privacy. But as far as I’m concerned, a typical downside is that the false positive rate will increase and there is no theoretical guarantee that the increase of false-positive rate will be negligible compared with the increase of true positive rate.
Its effectiveness in detecting backdoor attacks seems elusive. As we know, the backdoor attacks exist when users want to outsource the task of training the network to a third-party, which may potentially be an attack. Therefore, the training process is out-of-control to the detector. However, the paper proposes to use differential privacy to the model training process, which is not in the settings of a backdoor attack.

review_title: Official Blind Review #3

Meta-review: Thanks for the submission. This paper leverages the stability of differential privacy for the problems of anomaly and backdoor attack detection. The reviewers agree that this application of differential privacy is novel. The theory of the paper appears to be a bit weak (with very strong assumptions on the private learner), although it reflects the basic underlying idea of the detection technique. The paper also provides some empirical evaluation of the technique.
Decision: Accept (Poster)