Paper: LabelDP-Pro: Learning with Label Differential Privacy via Projections (ICLR-2024)

Reviewer comment (Theory):
I think the comparisons in the theoretical part need to be clear, i.e., what's the exact gain of the proposed method?

Author response:
We thank the reviewer for this question. The goal of the theoretical section is to study the theoretical convex setting to build intuition for the general case of deep neural networks (which are beyond the reach of current theoretical frameworks). The goal is to provide some intuition for how LabelDP-Pro with different denoisers might compare to each other; in fact, the analysis of this setting is precisely what guided us to try the different denoisers that we do, but this section is by no means supposed to a complete explanation for the empirical results. Note that we provide the following caveat in the last paragraph of Section 4, _“Note that the bounds in Table 4 hold only for convex loss functions. Moreover, they are upper bounds on the excess error. For instance, while the bound in Lemma 3 is tight in the worst case, better rates might be achievable under additional assumptions on the loss function. Despite these limitations, we see in Table 4 that the excess error bounds align well with the experimental findings.”_

Reviewer comment (Theory):
[Regarding the subtlety of privacy analysis of SELFSPAN]. If I understand it correctly, the problem is that even though the additional projection step does not touch sensitive data (labels) in this case, one cannot directly use post-processing. This is because the projection will leak the index of the sampling result, which then impacts the privacy amplification by subsampling in the original DP-SGD. Due to this, when using SELFSPAN, one cannot enjoy the gain/benefit of subsampling. On the other hand, since it does not touch labels, one can still use post-processing over the **non-subsampling** version of DP-SGD?

Author response:
Yes, indeed this is correct.

Label-DP Pro using Self-Span projection can be viewed as a post-processing of non-subsampling version of DP-SGD, whereas Label-DP Pro using Alt-Span denoiser can be viewed as a post-processing of the subsampled version of DP-SGD.
Perhaps a simple way to see this is as follows. Consider the following two mechanisms:
1. Mechanism $M_1$ that reveals, for each training step, just the noisy average gradient for the batch $I^G$ (without revealing $I^G$). Amplification by subsampling applies in this case.
2. Mechanism $M_2$ that reveals, for each training step, the entire _unlabeled_ batch $I^G$ along with noisy average gradients for that batch. Amplification by subsampling does not apply in this case.

DP-SGD and LabelDP-Pro with Alt-Conv denoiser can be viewed as a post-processing of $M_1$, since in the latter, the projection operation only uses non-private information, namely the alt-batch $I^P$.

On the other hand, LabelDP-Pro with Self-Span / Self-Conv denoisers cannot be seen as a post-processing of $M_1$, but can still be seen as a post-processing of $M_2$.

Reviewer comment (Evaluation):
[Regarding the improvement over DP-SGD] It seems to me that the main gain over DP-SGD is the improvement over dimension d? It might be better to give more discussions on Table 4, as there are different choices of \sigma. Being more specific or using more particular values will be better I think.

Author response:
We firstly note that there is a small typo in the bound for `AltConv`, namely, the $\sigma$ should be $\sigma_a$, which is the same $\sigma_a$ present in the case of DP-SGD (`NoOp`). We have fixed this typo in the updated submission.The $\sigma_{na}$ in the case of `SelfSpan` and `SelfConv` are the same, and larger than $\sigma_a$ by roughly a factor of $\sqrt{n / n_1}$.

If we compare the rows for `NoOp` and `AltConv`, we have indeed get rid of the dependence on $\sqrt{d}$ and replaced it by $\log^{0.25}(K n_2)$, which can be significantly smaller, although the denominator now is also a bit smaller ($\sqrt{T n_1}$ vs $\sqrt[4]{n_1}$). Thus, it is immediately clear that for a sufficiently large $d$, `AltConv` provides a better bound for any setting for parameters. We provide a more detailed comparison in Appendix E. As mentioned before we chose to keep the comparison qualitative, since as is common across deep learning, we use the theoretical convex setting to build intuition for the general case of deep neural networks (which are beyond the reach of current theoretical frameworks).

We can perform a rough estimation of the quantities involved as follows.
Firstly, note that $\sigma_a$ is to be chosen such that $T$ steps of DP-SGD with batch size $n_1$ (that is, sub-sampling probability of $n_1 / n$) is roughly given as $\sigma_a \approx (n_1 / n) \cdot \sqrt{T} / \varepsilon \cdot \log(T / \delta)$ (using the advanced composition theorem, and [amplification by subsampling](https://arxiv.org/abs/1807.01647)).

Suppose for concreteness, we chose $n_1 = n_2 = T = \sqrt{n}$ and using the bounds for $\sigma_a$ as above, we get the bounds in Table 4 (ignoring the $RC$ term) as
* for `NoOp` : $\sqrt{d} \cdot \frac{\log(n/\delta)}{\varepsilon n^{0.75}}$
* for `AltConv` : $\frac{\log^{0.25}(K n) \sqrt{\log(n/\delta)}}{n^{0.25}\sqrt{\varepsilon}}$

And thus, the bound for `AltConv` is better when $d \gtrsim \frac{\varepsilon n \sqrt{\log (K n)}}{\log(n/\delta)}$. Note that there is nothing special about the particular setting of $n_1, n_2, T$ here. For any setting of these parameters, there is a large enough $d$ for which the error bound for `AltConv` will be better, since it does not depend on $d$ at all.

Reviewer comment (Presentation):
[Confusion about the section name] For Section 4, the authors use SCO, which gives readers the impression that the goal is somehow the population excess loss. However, it turns out the authors are actually analyzing ERM. It might be better to replace the section name.

Author response:
We thank the reviewer for raising this point. We consider ERM only for simplicity so that we do not need to add additional assumptions (involving more parameters) about the inherent stochasticity in the gradients. We’ve renamed the section to reflect that we are mainly analyzing Convex ERM.

Reviewer comment (Experiments):
Both the proposed method and baseline numbers in the experiment can be potentially better. As they might be underestimated, the current comparison could be inaccurate, which is the main evaluation to illustrate the advantage of the proposed algorithm.

Author response:
Thanks for the comments! It is worth noting that the differences in accuracy between our results and those reported in previous studies can be attributed to the choice of a simpler convolutional network (see Table 10.b in Appendix D) as opposed to the ResNet used in earlier work. This variance in backbone models was driven by efficiency considerations, given our large-scale evaluations across multiple settings and benchmarks.

However, as our method builds upon DP-SGD and reduces the noise it injects; if the baseline DP-SGD accuracy improves, we shall be able to achieve better accuracy. For instance, as shown below, when we switch to a more complicated network (with 4 more Conv layers), DP-SGD, ALIBI and our method all achieve higher performance, while our method is still able to retain the improvement for small $\varepsilon$’s.

| $\varepsilon$ | ALIBI | DP-SGD | LabelDP-Pro |
|---|---|---|---|
| 0.05 | 9.9 | 10.3 | **18.4** |
| 0.1 | 9.9 | 28.9 | **31.5** |
| 0.2 | 22.4 | 36.0 | **38.3** |
| 0.5 | 46.3 | 48.3 | **50.1** |
| 1.0 | **68.7** | 51.6 | 52.3 |

Reviewer comment (Experiments):
The pure DP-SGD can be much better, which can potentially bring benefits for the proposed algorithm. For example, De et al., 2022 empirically show that DP-SGD can achieve 56.8% accuracy on CIFAR10 when $\varepsilon=1.0$, while the number reported in the paper is only 43.5%.

Author response:
Thanks for the comments! It is worth noting that the differences in accuracy between our results and those reported in previous studies can be attributed to the choice of a simpler convolutional network (see Table 10.b in Appendix D) as opposed to the ResNet used in earlier work. This variance in backbone models was driven by efficiency considerations, given our large-scale evaluations across multiple settings and benchmarks.

However, as our method builds upon DP-SGD and reduces the noise it injects; if the baseline DP-SGD accuracy improves, we shall be able to achieve better accuracy. For instance, as shown below, when we switch to a more complicated network (with 4 more Conv layers), DP-SGD, ALIBI and our method all achieve higher performance, while our method is still able to retain the improvement for small $\varepsilon$’s.

| $\varepsilon$ | ALIBI | DP-SGD | LabelDP-Pro |
|---|---|---|---|
| 0.05 | 9.9 | 10.3 | **18.4** |
| 0.1 | 9.9 | 28.9 | **31.5** |
| 0.2 | 22.4 | 36.0 | **38.3** |
| 0.5 | 46.3 | 48.3 | **50.1** |
| 1.0 | **68.7** | 51.6 | 52.3 |

Reviewer comment (Experiments):
When leveraging self-supervised learning with LabelDP, PATE-FM doesn't utilize the SelfSL on CIFAR-10, which seems a little unfair. It is reasonable to at least utilize a pretrained feature extractor as an initialization instead of random initialization when training teachers in PATE-FM. In this way, PATE-FM leveraging SelfSL is expected to have better accuracy.

Author response:
Thank you for raising this concern. We want to clarify that while PATE-FM does not incorporate Self-Supervised Learning (SelfSL), its Semi-Supervised Learning (SemiSL) pipeline achieves notably higher accuracy (95.5% non-private accuracy) compared to the SelfSL pipeline we used in our evaluation, which achieves 90.9% non-private accuracy. This observation suggests that their backbone model is, in fact, **stronger** than our model.

It is important to note that even though PATE-FM utilizes a more robust backbone model, our LabelDP-Pro approach continues to outperform PATE-FM in the high-privacy regime (see Table 6, $\varepsilon \leq 0.5$).

Reviewer comment (Novelty):
DP-SGD with gradient projection actually has been investigated for a while [1, 2] and not introduced in the paper. Although the novelty of the proposed method is acknowledged given that all previous labelDP algorithms are based on label flipping, it might be worthwhile to see the difference and comparison between the proposed method and the methods in the literature.

Author response:
Thanks for sharing the references. [1] proposes Gradient Embedding Perturbation (GEP), which first projects individual private gradients into a non-sensitive low-dimensional anchor subspace, and then perturbs the low-dimensional embedding and the residual gradient separately according to the privacy budget. However, we note that [1] requires a public dataset to compute their gradient subspace; while such data is possible for image classification tasks, it is typically **not available for recommendation tasks**, which is one of the major applications of Label DP.

That being said, we still present the results of [1] in the Table below for the review’s reference. As shown, [1] achieves similar performance to our methods: our LabelDP-Pro is better for smaller eps while [1] is better for larger ones. 

| $\varepsilon$ | DPSGD | GEP [1] | LabelDP-Pro (Ours) |
|---|---|---|---|
| 0.05 | 10.1 | 14.8 | **16.0** |
| 0.1 | 24.6 | 25.9 | **27.0** |
| 0.2 | 29.8 | 29.9 | **30.8** |
| 0.5 | 36.9 | 38.7 | **40.2** |
| 1.0 | 43.5 | **46.2** | 44.8 |

[2] introduces a projection step to maintain the trajectory of the descent algorithm within the gradients' subspace. While this theoretically suggests faster convergence rates, empirical studies are not presented in the original work, making it challenging to establish a direct comparison.

For completeness, we’ve updated our submission to discuss both papers as well (Appendix A).

Reviewer comment (Evaluation):
The main issue with applied DP is that production use cases really crave performances close to non-private regimes. Label private settings are very important for ads and marketing use cases and accuracy is of utmost value in these settings. Developers will prefer a medium privacy regime with very close to non-private utility compared to a highly private regime with very low utility.

Author response:
We appreciate this comment. We have explained why we consider the high-privacy regime in our [general response](https://openreview.net/forum?id=JnYaF3vv3G&noteId=6bj5h7E7Kl). In short, this consideration results from the intrinsic definition of DP and practical requirements for stronger privacy guarantees, as well as the unsatisfactory performance of existing LabelDP baselines.

Furthermore, we note that in user-level privacy applications (as detailed in Table 7), our approach continues to outperform the baseline for medium-level privacy with $\varepsilon=5$, which demonstrates the versatility of our method in delivering competitive utility even in scenarios where a moderate level of privacy is required.

Reviewer comment (Experiments):
I would like to see results of the proposed algorithm for epsilons between 1 and 10 too (for a fairer comparison and decision making for interested users).

Author response:
Thank you for your suggestion. Please find below the results for $\varepsilon$ values ranging from 0.05 to 10, with the best accuracy for each $\varepsilon$ highlighted in **boldface**. However, it is important to note that our method primarily focuses on achieving high privacy (i.e., the low-$\varepsilon$ regime). As a result, the relatively lower performance in larger $\varepsilon$ values is expected and has been explained in the theoretical analysis presented in Section 4 of our submission.

| $\varepsilon$ | RR | RR-Debiased | LP-2ST | ALIBI | DP-SGD | LabelDP-Pro (Ours) |
|---|---|---|---|---|---|---|
| 0.05 | 10.0 | 10.0 | 10.0 | 9.9 | 10.0 | **16.0** |
| 0.1 | 10.0 | 10.0 | 10.0 | 9.9 | 24.6 | **27.0** |
| 0.2 | 10.0 | 10.0 | 10.0 | 17.9 | 29.8 | **30.8** |
| 0.5 | 10.0 | 10.0 | 35.8 | 34.4 | 36.9 | **40.2** |
| 1.0 | 17.0 | 10.0 | **60.9** | 51.3 | 43.5 | 44.8 |
| 2.0 | 77.6 | 58.8 | 63.8 | **64.2** | 51.4 | 52.0 |
| 5.0 | 80.4 | 80.2 | 81.7 | **82.3** | 59.5 | 59.8 |
| 10.0 | 81.5 | 81.8 | 83.5 | **83.5** | 63.6 | 63.7 |

Reviewer comment (Experiments):
I would like to see an empirical epsilon study (like the one referenced by authors, e.g. Malek et al. 2021).

Author response:
Thank you for your suggestion. While we acknowledge the importance of privacy auditing and empirical privacy studies, it appears to be outside the scope of our current work. We intend to consider this for future research endeavors.

Reviewer comment (Theory):
The user-level private algorithm only leverages group privacy, which is extremely sub-optimal even in the simple problem of mean estimation. A good user-level private algorithm should go beyond the simple application of group privacy.

Author response:
We appreciate this comment. We would like to note that we ensure a fair comparison among all methods by **employing the same group privacy rule for all approaches**. Furthermore, we evaluated our approach not only with user-level privacy (Section 6), we also showed that it consistently outperforms the baselines in item-level privacy settings (Section 5).

We would also like to note that, while there are many recent results on user-level privacy that go beyond group privacy [1,2,3], none of the results are practical as far as we can tell (e.g. in the use of the i.i.d. or other assumptions on the data). This is demonstrated by the lack of empirical evaluations in these works. We would appreciate it if the reviewer could provide further clarification or pointers to prior work on practical user-level privacy that goes beyond simple group privacy.

[1] Daniel Levy, Ziteng Sun, Kareem Amin, Satyen Kale, Alex Kulesza, Mehryar Mohri, Ananda Theertha Suresh. “Learning with User-Level Privacy.” https://arxiv.org/abs/2102.11845 

[2] Esfandiari, Hossein, Vahab Mirrokni, and Shyam Narayanan. "Tight and robust private mean estimation with few users." https://arxiv.org/abs/2110.11876

[3] Badih Ghazi, Pritish Kamath, Ravi Kumar, Raghu Meka, Pasin Manurangsi, Chiyuan Zhang. “On User-Level Private Convex Optimization.” https://arxiv.org/abs/2305.04912

Reviewer comment (Evaluation):
For $\varepsilon \ge 1$ in Tables 5 and 6, the proposed algorithm underperforms the best baseline by a large margin. In practice, it is still reasonable to use $\varepsilon < 5$, so it would be better if the algorithm could also perform well for moderate level of privacy.

Author response:
We appreciate this comment. We’ve explained why we consider the high-privacy regime in our [general response](https://openreview.net/forum?id=JnYaF3vv3G&noteId=6bj5h7E7Kl). In short, this consideration results from the intrinsic definition of DP and practical requirements for stronger privacy guarantees, as well as the unsatisfactory performance of existing LabelDP baselines.

Furthermore, we note that in user-level privacy applications (as detailed in Table 7), our approach continues to outperform the baseline for medium-level privacy with $\varepsilon=5$, which demonstrates the versatility of our method in delivering competitive utility even in scenarios where a moderate level of privacy is required.

Reviewer comment (Evaluation):
When using pre-trained weights from SSL, the result for $\varepsilon > 1$ is much better. Maybe this is due to a more accurate gradient subspace after SSL pretraining?

Author response:
Thank you for this question. Yes, the accuracies achieved under SSL (Table 6) are higher than those achieved when trained from scratch (Table 5), which could be attributed to better image representations learnt by SSL and thus resulting in a more accurate gradient subspace.