Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes a general methodology to empirically evaluate the privacy of differentially private machine learning implementations, combining improved privacy search and verification methods with a toolkit of influence-based poisoning attacks. The paper proposes specific data poisoning schemes for algorithms like Logistic regression, random forests, and Naive Bayes.
strengths_and_weaknesses: Strengths:

1. I found the idea of estimating the probability over general model spaces as opposed to the probability over a single output to be interesting.

2. I also found the usage of local geometry for logistic regression to estimate the poisoning example to be very interesting.

Weaknesses:

1. The framework of auditing seems to be identical to prior work. So, the paper may have oversold the contribution.

2. Some of the improvements over prior work are specific to the learning tasks. (For example, the exact choice of influence function for logistic regression.)

3. I found the result for objective perturbation to be rather surprising. The fact that ML audit matches the theoretical bound is a bit counter intuitive. The reason being that in objective perturbation, typically the epsilons are split to ensure that both the ratio of Jacobians, and the log offs over the model outputs do not change by much. Unless the split is tuned for the best poisoning attack, I cannot intuitively see the curves to match. 

4. Are the results for objective perturbation specific to pure \epsilon DP? If not, then one has to use the bounds from Keifer et al. for the Gaussian distribution.
questions: Can you clarify (3) and (4) in the weakness.
limitations: Some of the improvements over prior work are specific to the learning tasks. (For example, the exact choice of influence function for logistic regression.)

The empirical results seem to be specific for delta=0 setting.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposed ML-Audit, a framework for estimating the differential privacy of a ML model. It is a general methodology to empirically evaluate the privacy of differentially private machine learning implementations, combining improved privacy search and verification methods with a toolkit of influence-based poisoning attacks. ML-Audit provides a recipe for devising audits against arbitrary models and is often orders of magnitude more effective than existing approaches.
strengths_and_weaknesses: Strengths:
1. The idea of proposing a framework for estimating the differential privacy of an ML model is quite novel and significant, as it is more universal with qualitatively improved performance.

2. The experiment session is solid, since this paper assesses Naive Bayes, logistic regression, and random forest on 6 common machine learning datasets and is compared with several strong baselines.

3. DP-based machine learning methods are theoretically rigorous approaches that provide a worst-case privacy guarantee for ML algorithms, which would have broad application prospects in many aspects, such as being used for practitioners to estimate the actual risk they incur in practice.

Weaknesses:
1. We suggest adding deep learning methods as the base models to the experiment, like DP-SGD, mentioned in line 34. Applying the proposed methods to deep learning models makes the paper more persuasive and valuable.

2. In Figure 2 (c), some results of OURS(ML-Audit) do not excel Swap-X baseline, and Swap-X performs similarly to OURS(ML-Audit) in output-perturbed logistic regression on linearly separable datasets such as iris. It will be better to give a corresponding explanation for such a case.

questions: Nothing to ask
limitations: Nothing
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper considers the problem of auditing machine learning models. They propose strategies to improve upon existing machine learning auditing approaches, and design poisoning attacks to audit a wider range of model types than have been previously considered in the auditing literature, such as Naive Bayes, logistic regression, and random forests. They show that their approach can be used both for bug finding and for understanding properties of a dataset which impact empirical privacy.
strengths_and_weaknesses: Strengths:

Using confidence intervals for the ratio of binomials is a nice improvement.

The paper designs model-specific poisoning attacks that work quite well. DP-SGD is not the only interesting DP learning algorithm, so this is a well motivated problem.

I liked the discussion of the impact of k.

Weaknesses:

The paper overclaims on the novelty of the framework. It is not very different from existing techniques. The main differences are the combination of DP-Sniper auditing with ML poisoning as proposed by Jagielski et al and Nasr et al, as well as the use of binomial ratio confidence intervals. The main contribution of the paper seems to be in the instantiations of the framework rather than the framework itself.

Strange choices of baselines. In the Jagielski et al. paper, ClipBKD is designed specifically for DP-SGD. It's a stretch to use it to audit objective perturbation, let alone Naive Bayes. That paper also used backdoor poisoning attacks. Nasr et al. have an optimization procedure that could be used to get model-adapted poisoning attacks. These are all appropriate baselines. What would be even cooler (and probably a contribution in its own) is taking advantage of the large literature on poisoning attacks (including specifically on objective perturbation and output perturbation for logistic regression [r1], nonprivate Naive bayes [r2]). These are generally not designed for auditing, so an auditing-specific poisoning strategy should still outperform them.

The motivation for dataset-specific privacy does not seem to be supported by the experiments. The authors want to investigate the idea that "achievable privacy is dataset-dependent", but the investigation focuses on a comparison between influence-based poisoning and ClipBKD. The achievable privacy is better measured by something like the maximum epsilon obtained over all attacks, not a comparison between attacks. What might help better make this point is investigating whether the "Ours" influence metric the authors consider in Figure 3b is correlated with the empirical epsilon lower bound, when analyzed across datasets (rather than across attacks on the same dataset). This might help give a cool metric to predict privacy risk before going through the auditing process.

[r1] - https://www.ijcai.org/proceedings/2019/0657.pdf
[r2] - https://people.eecs.berkeley.edu/~adj/publications/paper-files/SecML-MLJ2010.pdf
questions: Is the influence metric correlated with the gap between theoretical and empirical epsilon across datasets?

Did you ever experiment with baselines besides flipping and ClipBKD, something like standard backdoor attacks or the Nasr et al strategy?
limitations: This is fine.
review_title: null

Meta-review: metareview: The paper got overall positive reviews. I recommend the authors to carefully incorporate all the pointes raised during the rebuttal period. especially, in regards to the discussion on novelty. 
confidence: Certain
Decision: Accept