Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a method to help the developer to debug the implementation of SD-SGD. Three kinds of bugs can be theoretically guaranteed to be detected: 1. clipping to mini-batch gradient   2. no gradient clipping     3. noise without calibration. 
strength_and_weaknesses: Strength: The experiment is solid and sufficient

Weakness: I am not persuaded that the studied problem is important
clarity,_quality,_novelty_and_reproducibility: Clarity: Good

Quality: Okay, the problem studied may not be very important

Novelty: Okay, but not extremely novel 

Reproducibility: N.A., I am not an expert on NLP/CV. Thus, I have no idea whether the experiments can be reproduced or not.
summary_of_the_review: I don't think the studied problem is very important. Since there are already standard/official packages for DP-SGD or other DP optimizers, I don't believe getting DP-SGD incorrectly implemented is a common problem nowadays. Two examples of those packages are TensorFlow Privacy (https://github.com/tensorflow/privacy) and Opacus for PyTorch (https://opacus.ai/).
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a practical dev-level method for debugging differentially private SGD (DP-SGD), so that practitioners can check their implementations, and make sure all the steps of DPSGD  are working correctly and that the guarantee is achieved. Unlike the direction of work that does checking by finding bounds through attacks, this work focuses on the implementation and the training. More specifically, the proposed method checks the following three things a) clipping is happening, b) clipping is happening on a per-example level rather than a minibatch level c) noise addition is being done correctly. They do the first two checks by looking at the difference in loss by changing the clipping factor (for a) and minibatch size (b). They do the last check by training models multiple times and finding the standard deviation between the final parameters of the different models (looking at the l2 distance between model parameters).
strength_and_weaknesses: Strengths:

1. I think the problem of debugging DP implementations and having appropriate tests is really important, and practitioners could really benefit from built tools that could easily test their implementations. 



Weaknesses:

1. I think as this paper is more on the development/coding side, it is really important to have the code but I could not find a link to the anonymized repository. In general, how the tests/checks are implemented, how easy they are to use and how efficient they are is super important, as the checks themselves are very easy to come up with. The first two, i.e. changing clipping factor and batch size is something that I have done before. Another common thing people check is changing the epsilon value and setting it to very high and seeing how the performance improves. The last check however is novel and interesting. 

2. One thing that should be better clarified in the paper and treated with caution is that the first two checks should really not be done on actual private, eyes-off data! clipped gradients are still private and anything that is a function of them is off limits. So anything prior to noisy-ing the gradients cannot be seen. However, the good news is any data (even fake synthetic) would work here. I think this should be specified in the paper, otherwise there is privacy violations.

3. I worry about the efficiency of the method as for the checks things need to change and model needs to be trained multiple times. This could have a lot of overhead and it needs further analysis.

4. This is minor: I think the first 3 pages of the paper are very repetitive, without providing much information. This is especially the case for the intro. I think it would be better if the intro actually gave an overview of how you do the check, rather than repeating the stuff that is being checked. 
clarity,_quality,_novelty_and_reproducibility: The paper is clear and it has good quality. The novelty is mostly in the checking of the added noise. The results seem reproducible.
summary_of_the_review: I like the idea of having implementation checks for DP-SGD. Although I find the first two checks a bit obvious, I find the third one novel and I think having an implementation of them that can be easily used is really important.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper investigates the problem of finding out whether an implementation of DP-SGD is subject to silent bugs affecting the privacy protection, focusing on (i) gradient clipping and (ii) noise scaling. For each case, it proposes simple tests that relying on varying DP hyperparameters (batch size, noise level, clipping bound). The tests only assume the knowledge of a model training function function (black-box), which makes them agnostic and generic. The resulting tests are empirically validated on several datasets and architectures.
strength_and_weaknesses: # Strengths

- (Impact) This paper will be appealing for DP practitioners, as it addresses a real need to ensure the privacy of actual implementations of DP. I think this goes in a much-needed direction.
- The proposed tests are simple conceptually, yet effective for (i) and (ii), lightweight, and can be plugged over any black-box implementation of DP (agnosticism)

# Weaknesses

- (Clarity) The paper would benefit from a more formal writing: some hypotheses are buried within the text, others are missing, and the theoretical justification of the noise calibration test for DP-Adam lacks rigor (see more detailed comments below).
- (Limitations) A more minor weakness is that not all potential pain issues of DP implementations are covered, notably batch sampling issues: the paper is limited to noise calibration and gradient clipping
clarity,_quality,_novelty_and_reproducibility: # Clarity

- The paper is overall clear and one understands the main message.
- However, I think that the tests could be formalized a bit more, in particular regarding the assumptions that are made on the black box. This would make it easier to understand the conditions of validity of the proposed tests.
  - An example of such a constraint is in Sec 3.1, 
>Note that we would like our test to be implementable without having to make modification to the training script

  I think that summarizing all these assumptions in a dedicated section would help clarify the paper
  - Another example is the level of control the developer has on the samples provided to the black-box: is it controllable? Some sections hint at a positive answer, but I would encourage the authors to explicit it.
  - In sec 3.1, to detect mini-batch gradient clipping, the test relies on creating a specific batch of size $B$ such that $B-1$ samples have zero gradient and $B$ has a large gradient. If $\ell(\hat{y}, y)$ is the loss between the predicted $\hat{y} = M(x)$ and the true label $y$, the method proposed by the paper implicitly assumes that $\nabla_{\hat{y}} \ell (\hat{y}, \hat{y}) = 0$. This holds for the cross-entropy and the mean square error, but not necessarily for arbitrary losses.
- I think the abstract and the conclusion are slightly misleading as regards the mistakes actually covered by the paper. The sentences
> These mistakes include improper gradient computations or noise miscalibration

  (abstract) and
  > We are able to detect and identify common mistakes like incorrect gradient clipping and improper noise calibration

  (conclusion) wrongly give the impression that gradient computations and noise calibration are a *strict subset* of the properties of DP-SGD tested here. In fact, they are the only two. I think clarifying this contribution (which is perfectly valid on its own) would be less misleading regarding the actual paper content.

## Minor typos and comments
- Weights of the model are denoted $W$ L8 of Algorithm 1, but actually written $M$ in Eq (3) and other main text references
- In Eq (6), a subscript $g_b$ is missing to $g$
- In the proof of theorem 1, the last paragraph should probably not be included in the proof itself but after (QED symbol ill placed)

# Quality

- The proposed tests for the gradient clipping (i) are sound to me in the case of DP-SGD.
- Similarly, the proposed tests for noise calibration (ii), although slightly less clear than Sec 3.2, also seem relevant for DP-SGD
  - However, I don't understand why there is necessarily a need to directly look at the models; couldn't one use the losses, as in the case (i)? The need for performing long training steps ($100$ updates) is not clear to me either. For instance, for DP-SGD, for a fixed data batch and weights, one could do multiple single steps for different values of $C$ and check if the loss variation's variance depends on $C$. This question is also related to what the experimenter has access to / preferably accesses with respect to the model training loop.
- The validity of the tests proposed to address (ii) in the case of DP-Adam is not entirely clear to me. Indeed, Adam implies a renormalization, so it is unclear why the results should depend on $C$. For instance, in Fig 8 in Appendix E, one sees that the distance between models saturate: is this result expected?

# Novelty

The proposed approaches are novel to the best of my knowledge.

# Reproducibility

- In sec 3.1, the authors introduce a parameter $\alpha \gg 1$ to ensure having a "large enough" gradient. How is chosen $\alpha$?

summary_of_the_review: This paper addresses a practical problem of large importance and proposes simple yet effective methods. However, the paper could benefit from an improvement in writing, and the applicability of the tests to DP-Adam would require more theoretical justifications. If the authors addressed these concerns, I would be willing to significantly increase my score, as I think this paper could be impactful for the community.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper introduces light-weight approaches for debugging differentially private stochastic gradient descent (DP-SGD). The authors first identify the possible bugs which invalidate DP-SGD privacy guarantees (per-example clipping, noise calibration), and then proposes tests that aim to identify the exact problem (no gradient clipping, not clipping per-example grads; no noise, wrong noise scaling) without having to modify the source code. The authors empirically test their proposed solution to show that it correctly identifies various implementation issues in DP-SGD.

strength_and_weaknesses: ### Strengths:

1) The paper is generally very easy to follow and nice to read.

2) The proposed approach seems solid.

3) Debugging DP software can be really painful, since often the main signal of potential problems is that everything works too well. Good debugging tools are potentially very helpful.

### Weaknesses:

1) My main (and pretty much only) problem with the paper is that the current problems it aims to solve are quite specific (only DP-SGD) and not too hard to diagnose without the proposed tools. I would be more enthusiastic if the authors can expand the work to provide a framework for debugging DP software (at least somewhat) more generally. I do not believe that the current work warrants a publication in ICLR.


clarity,_quality,_novelty_and_reproducibility: The paper is clear and well-written. The proposed approach is novel, although limited in scope. I believe the main results from the paper are easy-enough to reproduce.

summary_of_the_review: A nice paper to read, but the current problem is too simple and lacks ambition.


### Update after rebuttal

After reading the other reviews and the author responses, I keep my score unchanged and recommend rejection: as I commented to the authors, I think debugging DP-SGD is simply not important enough to warrant publication in ICLR. If I would be to review this paper again in the future, the most important update that would persuade me to increase my score would be to try and make the debugging tools more widely applicable.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a methodology for developers to check for mistakes when implemented differentially private ML methods. The reviewers agree that this is an important question, perhaps more to practitioners, but there is not enough interest as the theory is minimal (and at times hard to read) for this to be a paper in top ML conference.
justification_for_why_not_higher_score: Seems below the borderline really.
justification_for_why_not_lower_score: NA
Decision: Reject