Paper: Beyond Laplace and Gaussian: Exploring the Generalized Gaussian Mechanism for Private Machine Learning (ICLR-2024)

Reviewer comment (Theory):
The paper's "Privacy Accounting for GG Mechanisms" relies heavily on numerical experiments without robust error control or rigorous proofs. This methodology is particularly concerning since the privacy profile of this noise family is already well-established in the literature.

Author response:
`` The paper's "Privacy Accounting for GG Mechanisms" relies heavily on numerical experiments without robust error control or rigorous proofs.``
* We address the error of the privacy accountant in Appendix B.3. More specifically, Theorem 6 provides an error analysis of the privacy accountant, which provides an arbitrarily tight privacy accountant. (We set our acceptable tolerance for \epsilon to be 0.01, and round our reported values up to the nearest 0.01.)

Reviewer comment (Experiments):
The experiments (PATE, DP-SGD) conducted under the privacy budget of \(\delta=10^{-5}\) are too limited to convincingly demonstrate the superiority of any specific \(\beta\) values. At this \(\delta\) setting, the noise variance added by the mechanisms varies widely, which is very likely to be the true reason behind the differing outcomes.

Author response:
``The paper's focus on a single privacy budget ((\delta=10^{-5})) is both arbitrary and unconvincing. For instance, releasing a (10^{-5}) portion of the total data satisfies (\delta=10^{-5}) with (\epsilon=0), making it theoretically superior to all other mechanisms discussed in terms of this metric…. The experiments (PATE, DP-SGD) conducted under the privacy budget of (\delta=10^{-5}) are too limited to convincingly demonstrate the superiority of any specific (\beta) values. At this (\delta) setting, the noise variance added by the mechanisms varies widely, which is very likely to be the true reason behind the differing outcomes. ``
* We thank you for this comment, we actually run our DPSGD experiments with delta=1e-6. However, it appears that in our paper, we wrote 1e-5 in one place (Appendix E.2), as a vestige of a previous version. We have updated this, thank you for catching this.
* It’s worth also noting that presenting results for a single choice of delta= 1e-5 is standard for many DPSGD experiments. Two well-cited and well-regarded papers that achieve SOTA results use delta=1e-5 for CIFAR-10  https://arxiv.org/abs/2011.11660  and ​​https://arxiv.org/abs/2204.13650. 
* Regarding the comment that at this delta, the variance varies widely. We are unsure what the reviewer is referring to, as the PRV accountant is correct to arbitrary precision.

Reviewer comment (Theory):
For a fixed \(\epsilon\), the Gaussian mechanism will perform poorly for very small \(\delta\) values as the required variance diverges rapidly. In contrast, for the Laplace mechanism, the scale of the Laplace noise remains approximately unchanged. Conversely, when considering Gaussian Differential Privacy (GDP) as the privacy budget, the Gaussian mechanism generally outperforms the Laplace mechanism. There is no intrinsic advantage or disadvantage for either of these two algorithms; their efficacy largely depends on the specific form of the privacy constraint. The statement "This provides a justification for the widespread adoption of the Gaussian mechanism in DP learning" seems coincidental within the scope of this research approach. The preference for the Gaussian mechanism may be more attributed to its strong alignment with GDP accounting, rather than any inherent superiority deduced from the methods used in this research.

Author response:
``There is no intrinsic advantage or disadvantage for either of these two algorithms; their efficacy largely depends on the specific form of the privacy constraint. The statement "This provides a justification for the widespread adoption of the Gaussian mechanism in DP learning" seems coincidental within the scope of this research approach.’’
* In Machine Learning applications of DP, there is an explicit advantage to choosing, e.g., Laplace over Gaussian noise because in practice they have different privacy-utility functions. PATE (https://openreview.net/pdf?id=rkZB1XbRZ) for example, explicitly cites the differences in privacy-accuracy tradeoffs and specifically the likeliness-of-outliers (for the same RDP accountant) as the reason for switching from laplace to gaussian noise. 
* For a particular delta and composition count, one incurs a particular tradeoff in weight-in-tails vs weight-in-body; our findings show that surprisingly, this does not contribute to a noticeable effect in DPSGD. See Appendix B.5 for one concrete measure of how this tradeoff occurs.

``The preference for the Gaussian mechanism may be more attributed to its strong alignment with GDP accounting, rather than any inherent superiority deduced from the methods used in this research. ``
* The PRV accountant is not designed with a particular mechanism in mind, and as far as we know there is no connection between GDP and PRV accounting. 
* Further, in the PRV paper (https://arxiv.org/pdf/2106.02848.pdf) they show that GDP accounting regularly underestimates privacy (as opposed to the PRV accountant).

Reviewer comment (Theory):
It is incorrect (vastly loose) to compute composition privacy budget from a single pair of epsilon and delta (think about composition of Gaussian mechanisms).

Author response:
``It is incorrect (vastly loose) to compute composition privacy budget from a single pair of epsilon and delta (think about composition of Gaussian mechanisms).``
* We agree. Is there a particular location where the reviewer sees this to be a problem? If the reviewer is referring to the fact that we produce a privacy curve for the privacy consumed in Figure 1, then we emphasize that this plot is there for illustrative purposes and that this plot would be qualitatively the same for a different choice of composition of (epsilon, delta) pairs (and not just a single pair).

Reviewer comment (Experiments):
In Section B.2, titled "Mechanisms with Equivalent Privacy Guarantees," the selection of \(\sigma\) is derived through random search, not through analytical computation. This approach does not guarantee the avoidance of numerical stability issues, which is very likely to happen for very small \(\delta\) when using the PRV accountant's random search to determine \(\epsilon\). The method should at least be executed using well-established mechanisms (Gaussian and Laplace) to validate the outputs against their analytically true values.

Author response:
``In Section B.2, titled "Mechanisms with Equivalent Privacy Guarantees," the selection of (\sigma) is derived through random search, not through analytical computation.``
* There is a misunderstanding here: we propose selecting sigma through a binary search rather than a random search. Binary search is a reasonable approach here because for fixed \beta and \delta, increasing \sigma increases \epsilon, making the privacy-sigma curve monotonic. Again we emphasize that the privacy accountant computes epsilon with arbitrary precision (shown in B.3).
Further, in practice we set (beta, sigma, sample rate q, compositions N, delta) and then compute the privacy accounting for that, rather than fixing epsilon, and solving for sigma.

Reviewer comment (Writing):
Some citations in this paper can be improved to be more relevant. For example, (Kairouz, P., Oh, S., & Viswanath, P. (2015)) as optimal composition for epsilon, delta DP is more suitable than (Abadi et al., 2016) which is not directly related to the topic after definition 4.

Author response:
Regarding the concrete points on citations and language, we thank you for these comments and introduce them now.

Reviewer comment (Novelty):
The proposed mechanisms ($\beta$-DP-SGD and GGNMax) are a straightforward generalization of the existing mechanisms by replacing noise distribution.

Author response:
``The proposed mechanisms (beta-DP-SGD and GGNMax) are a straightforward generalization of the existing mechanisms by replacing noise distribution.``
* While we agree that there are many new mechanisms to explore, to the best of our knowledge, our work is the first to study privacy-accuracy tradeoffs with alternative noise distributions for DPSGD or PATE. The GG family is a natural place to start, because it contains both Laplace and Laussian mechanisms as special cases, but should be seen as the start of a line of work, not the end.
* We view the relation to existing mechanisms as a strength rather than a weakness. Our results show that even minor changes to existing commonly-used algorithmic structures can provide empirical improvements, and that existing private ML algorithms can be easily extended to accommodate optimally-tuned noise from the GG mechanism.
*We emphasize that our main contributions are not novel algorithmic developments, but rather the analysis and application of the GG family of mechanisms to ML applications of interest through these existing algorithms (DP-SGD and PATE).

Reviewer comment (Novelty):
The analysis using the PRV accountant is not novel.

Author response:
```The analysis using the PRV accountant is not novel.```
* The PRV accountant is impressive work that provides a huge step forward in privacy accounting. However, prior to our work the PRV accountant could only be used with mechanisms with known privacy loss random variables, and their CDFs -- something that is generally non-trivial to compute. While it is intuitive to numerically compute this, we provide arbitrarily tight error bounds, showing that this is acceptable to do, and we provide the sample complexity of doing so. Further, in our new version of the paper, we provide a proof that the GG mechanism (with l_beta sensitivity) is dimension independent (in Appendix B.2), allowing for much easier accounting. To the best of our knowledge, both of these ideas are novel and not previously known.

Reviewer comment (Theory):
There are not many analytical results in the generalized Gaussian mechanisms; most of the results are empirical findings, and the experiments may not be sufficient to claim the findings.

Author response:
```I felt that there are not many analytical results in the generalized Gaussian mechanisms. Most of the results are empirical findings, and I doubt that the experiments are sufficient to claim the findings.```
* We agree that more theoretical guarantees would be nice but for both GGNmax and DPSGD this is problematic for two reasons. 1. The best privacy accountants are numerical, and thus prevent us from making a statement about a privacy-accuracy tradeoff. 2. For DPSGD it is exceedingly difficult to make theoretical statements about non-convex problems. These two reasons make it very difficult to make a theoretical statement. However, for the GGNmax setting, we attempt to get around this by providing numerical experiments from PATE (main body), simulations from a larger family of histograms (Appendix D.3), and theoretical analysis when using the RDP accountant (Appendix D.4).

Reviewer comment (Theory):
Is Theorem 1 saying there exists $\epsilon$ and $\delta$ for $(\epsilon,\delta)$-DP? Isn't it obvious that every mechanism has $\epsilon=1$ and $\delta=1$ if I am not mistaken?

Author response:
``Is Theorem 1 saying there exists eps  and delta  for (eps,delta)-DP? Isn't it obvious that every mechanism has eps=1 and delta = 1  if I am not mistaken? ``
* You are correct that if delta is allowed to be 1, then all mechanisms are (epsilon, delta)-DP; however Theorem 1 states that there is an epsilon value for any choice of beta, sigma, and delta. While this seems intuitive in some cases it is certainly not true for all algorithms, and this precondition is a prerequisite to using the PRV Accountant. As an example, a mechanism that runs the \epsilon-DP Laplace mechanism with probability 0.9, and with the remaining 0.1 probability returns the non-noisy answer is (\epsilon, \delta)-DP for some \delta, but not for any delta < 0.1.

Reviewer comment (Evaluation):
For "However, unlike the results ... larger than $\beta>3$" on page 8, is there any reason why this happens?

Author response:
``"However, unlike the results…. larger than beta> 3 " on page 8, is there any reason why this happens?``
* First, we emphasize that the label accuracy decline is on the order of ~0.1%, and so this is not a significant drop. The drop is most noticeable specifically because of how little variance there is in the label accuracy across values of \beta. That said, we believe the drop is due to an artifact of how the mechanisms of equivalent DP guarantees are generated. To describe it in detail: first we add noise from an evenly distributed grid of \beta \in [1,4] and \sigma \in [0.01,7] values, and then we compute the privacy levels for those (\beta,\sigma) tuples and compute the corresponding mechanisms of approximately equal DP guarantees. Because of this, we overestimate \epsilon when solving for mechanisms with equivalent DP guarantees; this is most noticeable for larger choices of \beta, where the same difference in \sigma contributes to a larger \epsilon.

Reviewer comment (Experiments):
If the comparisons regarding $\beta$ are done after hyper-parameter optimization, is it reasonable to perform the comparison by fixing hyperparameters that are optimal in non-DP training?

Author response:
``Is it reasonable to perform the comparison by fixing hyperparameters that are optimal in non-DP training? ``
* Previous work has shown that hyperparameters which are optimal in non-DP training are not necessarily optimal for DP training. This was explored in this work https://arxiv.org/abs/2007.14191, and a later work https://arxiv.org/abs/2110.03620 produced a scheme for how to do privacy preserving hyperparameter tuning.

Reviewer comment (Experiments):
For fixed hyperparameters, is $\beta=2$ still near-optimal, or is it dependent on hyperparameters?

Author response:
``For fixed hyperparameters, beta=2  is still near-optimal? or is it dependent on hyperparameters?``
* In general, it is near optimal, but it does depend on the hyperparameters, we provide plots for this in Appendix E.3.

Reviewer comment (Evaluation):
In the caption of Figure 4, it is written that the reported three epsilons are based on the minimum epsilon giving $0.98$ accuracy. Why is the accuracy much worse than $0.98$ in the plots even for $\epsilon^\prime$?

Author response:
```In the caption of Figure 4, it is written that the reported three epsilons are based on the minimum epsilon giving 0.98 accuracy. Why then is the accuracy is much worse than 0.98 accuracy  in the plots even for eps’?```
* In our submission, we caught that there was a potential improvement by switching from l2 to l_beta sensitivity. As such, we have regenerated Figure 4, and we show our plots for fixed epsilons. Thank you for catching this, and we hope the new changes address your concern.

Reviewer comment (Evaluation):
Claiming STOA with three runs: The results on machine learning had quite a high variance due to limited runs. I think claiming STOA when the accuracy is better by such a small amount is not significant. Perhaps more runs and a hypothesis test/confidence interval would give more definitive results (although STOA is not the primary goal of the work).

Author response:
Regarding the state-of-the-art results: as you stated, achieving state-of-the-art was not the primary goal of the work, and rather only a baseline to compare against; given this goal, we found that 3 runs were sufficient because the resulting standard deviations in nearly every case was less than 0.3%.

Reviewer comment (Novelty):
One more hyperparameter: the very small gains in accuracy would surely be outweighed by the cost of tuning beta in practice. I see the main contribution of this work is showing that the beta doesn't have too much effect rather than improving the accuracy of current mechanisms.

Author response:
We agree that this is one more hyperparameter to tune, and generally agree that “I see the main contribution of this work is showing that the beta doesn't have too much effect rather than improving the accuracy of current mechanisms.” We also agree, as you state, that “there are so many hyperparameters to tune, so giving practitioners support in using the Gaussian mechanism as a default is useful.”

Reviewer comment (Novelty):
While the study is fairly interesting, I'm not sure if the technical contribution is enough. The main takeaway is Gaussian and Laplace are pretty much the best choices as expected. Generalized Gaussian Mechanism is fairly hard to analyze since it doesn't have a closed-form relationship between $\beta$ and $\epsilon, \delta$. The experiment also requires the tuning of $\beta$ which in the end ends up being Gaussian and Laplace mechanism anyway.

Author response:
``While the study is fairly interesting, I'm not sure if the technical contribution is enough``
* Prior to our work, very little research has explored alternative differential privacy techniques for private machine learning. One notable exception is the Cactus mechanism (https://arxiv.org/abs/2207.00420), however, it is not implementable for models with a large number of parameters (>40). As far as we are aware, no work exists even considering Laplace noise for DP-SGD. Yes, our work finds that the Gaussian mechanism is close-to-optimal for the family of privacy mechanisms we explore. However, we emphasize our other contributions:
    * We provide the first exploration of DP mechanisms outside of the Gaussian Mechanism for DPSGD for private ML and provide a practical implementation of this mechanism for private ML.
    * We provide a framework for comparing Laplace and Gaussian mechanisms, as well as how to use existing privacy accountants for arbitrary mechanisms.
    * To us, the authors, it is exceedingly surprising that the Gaussian Mechanism performs close to optimally. It’s hard to apply a historical rationale for why previous researchers only considered the Gaussian mechanism, but it seems likely that this is because they were familiar with it mathematically, and it has nice properties for computing integrals. However, neither of those two reasons are any reason why the Gaussian mechanism (out of the infinitely many other possible mechanisms) should perform optimally. As a field, we have explored very little of the space of potential DP mechanisms, and to find that one of the two original mechanisms were in fact near optimal, is at the very least surprising. 
    * In our exploration we provide a _mostly_ negative result, that researchers should mostly not explore other unimodal DP mechanisms for DPSGD. While positive results are easier to accept, a negative result (that Gaussian noise performs near-optimally) is nonetheless a significant step forward in our understanding of privacy-accuracy tradeoffs in machine learning.

Reviewer comment (Theory):
In page 18, I think the RHS should be $(\frac{\alpha-1}{\alpha})^{1/\beta}$?

Author response:
Regarding your questions - yes, we believe you are correct, and we have updated it accordingly. As you noted, we also find that this does not change the argument.

Reviewer comment (Theory):
Also as $x<0$, shouldn't $\frac{|x|}{|x-\mu|}$ be $\frac{-x}{\mu-x}$ instead? Fortunately, I don't think it affects the argument.

Author response:
Regarding your questions - yes, we believe you are correct, and we have updated it accordingly. As you noted, we also find that this does not change the argument.