User-Level Differentially Private Learning via Correlated Sampling

Abstract: Badih Ghazi, Ravi Kumar, Pasin Manurangsi

Reviews and author responses:

 Thanks for your response. After the discussion period, my evaluation remains the same.

 Thank you for the detailed review and suggestions. We will consider incorporating the suggestions on rearranging the proofs/lemmas/definitions in the revision.

**Large number of samples per user.**
We do agree with the reviewer that this is an interesting open question and we indeed discuss it in the first paragraph of Section 5. Nonetheless, we would like to stress that even a characterization of the number of samples in the (simpler) item-level setting is not fully settled for $(\epsilon, \delta)$-DP setting; the current gap is between $O(\mathrm{poly LDim})$ and $\Omega(\log^* \mathrm{LDim})$, which is extremely large. As a result, we start by identifying the number of users required assuming that each user has sufficiently many samples.

**IID assumption.**
We are not aware of an extension of PAC learning to the heterogeneous setting. However, there have been some recent work in federated learning that consider *convex optimization* problems with heterogeneous distributions. We agree that this is a good avenue for future research and we discuss it in Section 5.


 Thank you for your review and pointing out the typos.  As you remark, requiring shared randomness is a limitation of our work and we will expand on that in the revision. 

 **Definition of $\alpha$.**
$\alpha$ refers to the error of the hypothesis $f$ output by the learner; we will define this appropriately.  

**Theorem 3 statement.**
Apologies and thanks for pointing this out.  Theorem 3 statement should refer to the 
*central* model.  


 **Large number of samples per user.**
We agree that this is an interesting open question and we indeed discuss it in the first paragraph of Section 5. Nonetheless, we would like to stress that even a characterization of the number of samples in the (simpler) item-level setting is not fully settled for $(\epsilon, \delta)$-DP; the current gap is between $O(\mathrm{poly~LDim})$ and $\Omega(\log^* \mathrm{LDim})$, which is extremely large. As a result, we start by identifying the number of users required assuming that each user has sufficiently many samples.


This paper considers the problem of private PAC learning in the setting of user-level privacy, that is, each user may have $m \ge 1$ samples. The authors consider three possible settings 1. approximate $(\varepsilon,\delta)$-DP, 2. pure $\varepsilon$-DP, and 3. Local DP. For each setting they show that in the user-level DP version ($m>1$), private learning needs smaller number of users compared to standard DP with $m=1$, with the caveat that each user is required to have sufficiently many samples (usually polynomial in the dimension $d$ and accuracy). For example, they develop an $(\varepsilon,\delta)$-DP algorithm that uses only $O( \log(1/\delta)/\varepsilon)$ users to learn any privately learnable class, where each user must have $m \ge \text{poly}(d)$ samples ($d$ is the Littlestone dimension of the class). They also provide tight lower bounds on the number of required users.  The paper considers an interesting problem and makes significant theoretical contributions. However, I have two main comments\concerns regarding this paper:

1. Motivation: as the authors mention in the intro, their main motivation for studying this problem is its relevance to the federated learning (FL) setting where each user contributes multiple samples. However, the setting of this paper is different than the standard FL setting in two ways:

      1.1 The number of users $n$ is (relatively) small while the number of samples per user $m$ can be too large (polynomial in dimension), in contrast to standard FL setting where $n$ is large and $m$ is smaller.  Currently, all of the algorithms in the paper crucially build on having large $m$ (to accurately estimate the hypothesis distribution), hence new techniques will probably be needed for the more relevant setting where $m$ is small. Moreover, in the setting of the paper, usually the number of samples per user $m$ is large enough that each user can learn on their own without using other users' data. It is therefore important to first highlight this in the paper and motivate this setting (which is different that the standard setting in FL in practice).
       
      1.2 I.I.D. samples: the authors assume all of the users have IID samples from the same distribution. I wonder whether these results can be extended to the heterogeneous setting where users have similar distributions (which is more relevant in FL).

It is therefore important that the authors motivate the setting where their theoretical results are obtained and explain its relevance.

2. Clarity: overall the paper is well-written, especially the intro where the authors clearly explain the problem and their main contributions. However, I think the technical sections need more work and organization to make the important ideas more visible. I suggest a few edits which can help for the reader:

      2.1 The proof of section 4.1 (from list-stability to pseudo-stability) includes many unnecessary lemmas that only include technical mathematical analysis (Lemmas 22-24). I suggest moving all of these to the appendix and instead adding a new lemma that has the consequence of these three lemmas: the distribution $\hat P_{H,G_1,\dots,G_k}$ is close to $P$. This lemma is more meaningful and gives some intuition for the design of the algorithm. Moreover, the proof becomes much more intuitive and easier to understand.

     2.2 Sections 2 and 3 include many definitions and lemmas which makes it extremely hard to read. Some parts of it are also used only at the end of the paper, making it challenging to recall some of the definitions. For example, it took me some time to find the definition of $c_h$ in equation $(2)$. It may be useful to reorganize these two section and maybe delay some of the definitions to the sections where they are used.

     2.3 It may be useful to start with the transformation from pseudo-stability to approximate DP of section 4.2 (instead of 4.1) as it's simple and gives a nice motivation for the definition of pseudo-stability.


 yes

This work considers the learnability problem under user-level differential privacy. The authors demonstrate that for any privately PAC learnable class, user-level DP algorithms require much fewer users compared to the item-level counterpart, as long as each user has sufficient samples. The main technique is a correlated sampling strategy that relies on a new generalization of global stability.  Strength:
This work gives a new understanding of learnability under user-level DP with strong results on the number of users needed to learn a concept class given sufficient samples per user. For approximate DP, the number of users only depends on the privacy and accuracy parameters. For pure DP in both the central and local model, nearly tight bounds on the number of users are obtained (the bound is tight for the central model). This work demonstrates a gap between $(\epsilon, \delta)$-DP and pure DP algorithms (including LDP), and also between user-level and item-level privacy. 

Weakness:
The main question is that this work assumes that the number of samples of each user is sufficiently large ($poly(d/\alpha)$ where $d$ is Littlestone or probabilistic representation dimension). I think the result would be more interesting if we could characterize learnability when the number of samples per user is fixed. Yes.

The authors show that any online learnable class can be learned with an epsilon delta DP algorithm using only O(log(1/delta)/epsilon) users as long as each user has enough samples. They show that there exists pure DP algorithm to learn any concept class that has finite probabilistic representation dimension in the local (public randomness) model. They also show that the bound can be improved w.r.t. the number of users in the central model of DP. They also show a lower bound on the number of users there must be for an accurate DP learning to exist. Finally, they show that Statistical Query algorithms can be turned into efficient learners.  Unfortunately I was not really qualified to review this paper, so I can not provide feedback on almost anything. What I could understand is that by generalizing the definition of global stability one can use public randomness to create private mechanisms to learn certain classes of hypothesis. So this work seems novel. I'm giving an accept but will have to trust the other reviewers on this one. My apologies.

Minor:

Line 53: what is \alpha?

Is Theorem 3 in page 2 stated correctly? Line 74 mentioned the central model but line 76 mentions the local model.

 NA

This paper studies a setting of differential privacy called user-level differential privacy. In this setting, each user has a set of samples and differential privacy is with respect to the addition or removal of a single user (instead of the classical definition where DP must hold with respect to the addition or removal of a single data point).

The authors prove a number of results. Their first result is that if a class is online learnable with Littlestone dimension $d$ then it is also $(\epsilon, \delta)$-DP learnable with $O(\log(1/\delta)/\epsilon)$ users and roughly $\text{poly}(d/\alpha)$ samples per user. The authors then go on to discuss further results using another notion of complexity called probabilistic representation dimension. The aforementioned results are not efficient and the authors give another result that is efficient by showing how an efficient statistic query algorithm can be transformed into an efficient learner in the user-level DP setting.

In terms of techniques, the authors use correlated sampling and build upon the work of Bun et al. and Ghazi et al. (on the equivalence of online learning and approximate differential privacy). With respect, to their result involving Littlestone dimension, the authors show a reduction from "list-global stability" to "pseudo-global stability" and then a (simple) reduction from "pseudo-global stability". List-global stability means that with high probability, given i.i.d. samples, an algorithm will output a list which contains a particular hypothesis. Using correlated sampling, they show that this can be leveraged to output a fixed hypothesis with high probability (which they call pseudo-global stability). Thus, by having all users share the same randomness, they can ensure that most users return a similar hypothesis with high probability and thereby using an approximate-DP selection algorithm to privately return a hypothesis.  **Strengths**\
The problem that the authors study is interesting, original, and well-motivated. User-level privacy is a relatively new topic (at least when studied from a theoretical viewpoint). In terms of techniques, the starting point appears to be the techniques of Bun et al. and Ghazi et al. for the equivalent of online learning and approximate differential privacy. However, the current paper builds upon these techniques in a nice and original way.

In addition, the paper is written quite well. The authors explain their approach clearly and the technical exposition is fairly clear as well.

**Weaknesses**\
I do not really have any complaints about the present paper although it would certainly be interesting to see if some of their results can be strengthened (for example, removing shared randomness).

**Target Audience**\
The paper will certainly be of interest to researchers working in the area of differential privacy. I also think that the paper sets up some interesting open problems for future work. For example, it would be interesting to see whether or not correlated sampling is required for their result.

**Minor Comments**\
-Lemma 13: Fix paranthesis inside the $(\cdot, \cdot)$-accurate. \
-Line 283: with probability $1-\beta^2/60$\
-Theorem 25: I believe the $(0.9, \beta/3)$-pseudo ... should be $(0.9, 1-\beta/3)$-pseudo...
 Yes

Meta-review: This paper considers the problem of private PAC learning in the setting of user-level privacy, where each user holds several labeled samples. They show that if every user holds sufficiently many samples, then very few users are needed to proved user-level privacy. They also provide lower bounds on the number of required users. The reviewers agree that this is an interesting paper with solid contribution, that should be accepted to NeurIPS.

Decision: accept