Differentially Private Sampling from Distributions

1 Introduction . Statistical machine learning models trained on sensitive data are now widely deployed in domains such as education , finance , criminal justice , medicine , and public health . The personal data used to train such models are more detailed than ever , and there is a growing awareness of the privacy risks that come with their use . Differential privacy is a standard for confidentiality that is now well studied and increasingly deployed . Differentially private algorithms ensure that whatever is learned about an individual from the algorithm ’ s output would be roughly the same whether or not that individual ’ s record was actually part of the input dataset . This requirement entails a strong guarantee , but limits the design of algorithms significantly . As a result , there is a substantial line of work on developing good differentially private methodology for learning and statistical estimation tasks , and on understanding how the sample size required for specific tasks increases relative to unconstrained algorithms . A typical task investigated in this area is distribution learning : informally , given records drawn i.i.d . from an unknown distribution P , the algorithm aims to produce a description of a distribution Q that is close to P . However , often the task at hand requires much less than full-fledged learning . We may simply need to generate a small amount of data that has the same distributional properties as the population , or perhaps simply “ looks plausible ” for the population . For example , one might need realistic data for debugging a software program or for getting a quick idea of the range of values in a dataset . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this work , we study a basic problem that captures these seemingly less stringent aims . Informally , the goal is to design a sampling algorithm that , given a dataset with n observations drawn i.i.d . from a distribution P , generates a single observation from a distribution Q that is close to P . To formulate the problem precisely , consider a randomized algorithmA : Un → U that takes as input a dataset of n records from some universe U and outputs a single element of U . Given a distribution P on U , let X = ( X1 , ... Xn ) be a random dataset with entries drawn i.i.d . from P . Let A ( X ) denote the random variable corresponding to the output of the algorithmA on input X . This random variable depends on both the selection of entries of X from P and the coins of A . Let QA , P denote the distribution of A ( X ) , so that QA , P ( z ) = Pr X ∼i.i.d.P coins ofA ( A ( X ) = z ) = ∑ x∈Un ( Pr ( X = x ) · Pr coins ofA ( A ( x ) = z ) ) . The dataset size n is a parameter of A and thus defined implicitly . We measure the closeness between the input and output distributions in total variation distance , denoted dTV . Definition 1.1 ( Accuracy of sampling [ 4 ] ) . A sampler A is α-accurate on a distribution P if dTV ( QA , P , P ) ≤ α . A sampler is α-accurate on a class C of distributions if it is α-accurate on every P in C. The class C in the accuracy definition above effectively encodes what the sampler is allowed to assume about P . For example , C might include all distributions on k elements or all product distributions on { 0 , 1 } d ( that is , distributions on d-bit strings for which the individual binary entries are independent ) . Our aim in formulating Definition 1.1 was to capture the weakest reasonable task that abstracts the goal of generating data with the same distributional properties as the input data . Without any privacy constraints , this task is trivial to achieve : an algorithm that simply outputs its first input record will sample from exactly the distribution P , so the interesting problem is to generate a sample of size m > n when given only n observations ( as in the work of Axelrod et al . [ 4 ] , which inspired our investigation ) . However , a differentially private algorithm can not , in general , simply output one of its records in the clear . Even producing a single correctly distributed sample is a non-trivial task . To understand the task and compare our results to existing work , it is helpful to contrast our definition of sampling with the more stringent goal of learning a distribution . A learning algorithm gets input in the same format as a sampling algorithm . We state a definition of distribution learning formulated with a single parameter α that captures both the distance between distributions and the failure probability . Definition 1.2 ( Distribution learning ) . An algorithm B learns a distribution class C to within error α if , given a dataset consisting of independent observations from a distribution P ∈ C , algorithm B outputs a description of a distribution that satisfies Pr X ∼i.i.d.P coins of B ( dTV ( B ( X ) , P ) ≤ α ) ≥ 1− α . An algorithm B that learns class C to within error α immediately yields a 2α-accurate sampler for class C : the sampler first runs B to get a distribution P̂ and then generates a single sample from P̂ . Thus , it is instructive to compare bounds on the dataset size required for sampling to known results on the sample complexity of distribution learning . Lower bounds for sampling imply lower bounds for all more stringent tasks , including learning , whereas separations between the complexity of sampling and that of learning suggest settings where weakening the learning objective might be productive . Differential privacy Differential privacy is a constraint on the algorithmA that processes a dataset . It does not rely on any distributional property of the data . We generally use uppercase letters ( e.g. , X = ( X1 , ... , Xn ) ) when viewing the data as a random variable , and lowercase symbols ( e.g. , x = ( x1 , ... , xn ) ) when treating the data as fixed . Two datasets x , x′ ∈ Un are neighbors if they differ in at most one entry . If each entry corresponds to the data of one person , then neighboring datasets differ by replacing one person ’ s data with an alternate record . Informally , differential privacy requires that an algorithm ’ s output distributions are similar on all pairs of neighboring datasets . Definition 1.3 ( Differential Privacy [ 15 , 14 ] ) . A randomized algorithm A : Un → Y is ( ε , δ ) differentially private ( in short , ( ε , δ ) -DP ) if for every pair of neighboring datasets x , x′ ∈ Un and for all subsets Y ⊆ Y , Pr [ A ( x ) ∈ Y ] ≤ eε · Pr [ A ( x′ ) ∈ Y ] + δ . For ε ≤ 1 and δ < 1/n , this guarantee implies , roughly , that one can learn the same things about any given individual from the output of the algorithm as one could had their data not been used in the computation [ 19 ] . When δ = 0 , the guarantee is referred to as pure differential privacy . 1.1 Our results . We initiate a systematic investigation of the sample complexity of differentially private sampling . We provide upper and lower bounds for the sample size needed for this task for three natural families of distributions : arbitrary distributions on [ k ] = { 1 , . . . , k } , arbitrary product distributions on { 0 , 1 } d , and product distributions on { 0 , 1 } d with bounded bias . We demonstrate that , in some parameter regimes , private sampling requires asymptotically fewer samples than learning a description of P nonprivately ; in other regimes , however , sampling proves to be as difficult as private learning . Our results are summarized in Table 1 . Proofs of all results are included in the supplementary material . For simplicity , the table and our informal discussions focus on ( ε , δ ) -differential privacy ( Definition 1.3 ) in the setting1 where δ = 1/nc for some constant c > 1 . Let Ck be the class of k-ary distributions ( that is , distributions on [ k ] ) . We show that n = Θ ( k αε ) observations are necessary and sufficient for differentially private sampling from Ck in the worst case . Theorem 1.4 . For all k ≥ 2 , ε > 0 , and α ∈ ( 0 , 1 ) , there exists an ( ε , 0 ) -DP sampler that is α-accurate on the distribution class Ck for datasets of size n = O ( kαε ) . Theorem 1.5 . For all k ≥ 2 , n ∈ N , α ∈ ( 0 , 150 ] , ε ∈ ( 0 , 1 ] , and δ ∈ ( 0 , 1 5000n ] , if there is an ( , δ ) -DP sampler that is α-accurate on the distribution class Ck on datasets of size n , then n = Ω ( k αε ) . The second major class we consider consists of products of d Bernoulli distributions , for d ∈ N. We denote this class by B⊗d . Each distribution in B⊗d is described by a vector ( p1 , ... , pd ) ∈ [ 0 , 1 ] d of d probabilities , called biases ; a single observation in { 0 , 1 } d is generated by flipping d independent coins with respective probabilities p1 , ... , pd of heads . We show that n = Θ̃ ( d ) observations are necessary and sufficient for differentially private sampling from B⊗d in the worst case . Theorem 1.6 . For all d , n ∈ N and , δ , α ∈ ( 0 , 1 ) , there exists an ( , δ ) -DP sampler that is α-accurate on the distribution class B⊗d for datasets of size n = Õ ( d α ) , assuming log ( 1/δ ) = poly ( log n ) . Theorem 1.7 . For all sufficiently small α > 0 , and for all d , n ∈ N , ε ∈ ( 0 , 1 ] , and δ ∈ [ 0 , 15000n ] , if there is an ( ε , δ ) -DP sampler that is α-accurate on the distribution class B⊗d on datasets of size n , then n = Ω ( d ) . Finally , we give better samplers and matching lower bounds for Bernoulli distributions and , more generally , products of Bernoulli distributions , with bias bounded away from 0 and 1 . For simplicity , 1This setting precludes trivial solutions ( which are possible when δ = Ω ( 1/n ) ) , but allows us to treat factors of log ( 1/δ ) as logarithmic in n and absorb them in Õ expressions . we consider distributions with bias pj ∈ [ 13 , 2 3 ] in each coordinate j ∈ [ d ] . For this class , we show that differentially private sampling can be performed with datasets of size roughly √ d/ε , significantly smaller than in the general case . Curiously , the accuracy parameter α has almost no effect on the sample complexity . For Bernoulli distributions with bounded bias , we achieve this with pure differential privacy , that is , with δ = 0 . For products of Bernoulli distributions , we need δ > 0 . Theorem 1.8 . For all d ∈ N and , δ , α ∈ ( 0 , 1 ) , there exists an ( ε , δ ) -DP sampler that is αaccurate on the class of products of d Bernoulli distributions with biases in [ 1 3 , 2 3 ] for datasets of size n = O ( √ d log ( 1/δ ) ε + log d α ) . When d = 1 , the sampler has δ = 0 and n = O ( 1ε + log 1 α ) . Theorem 1.9 . For all sufficiently small α > 0 , and for all d , n ∈ N , ε ∈ ( 0 , 1 ] , and δ ∈ [ 0 , 1100n ] , if there exists an ( ε , δ ) -DP sampler that is α-accurate on the class of products of d Bernoulli distributions with biases in [ 1 3 , 2 3 ] on datasets of size n , then n = Ω ( √ d/ε ) . Implications Our results show that the sample complexity of private sampling can differ substantially from that of private learning ( for which known bounds are stated in Table 1 ) . In some settings , sampling is much easier than learning : for example , for products of Bernoulli distributions with bounded biases , private sampling has a lower dependence on the dimension ( specifically , √ d instead of d ) and essentially no dependence on α . Even for arbitrary biases or arbitrary k-ary distributions , private sampling is easier when α ε . In other settings , however , private sampling can be as hard as private learning : e.g. , for ≤ α , the worst-case complexity of sampling and learning k-ary distributions and product distributions is the same . A more subtle point is that , in settings where private sampling is as hard as private learning , sampling accounts for the entire cost of privacy in learning . Specifically , the optimal sample complexity of differentially private learning for arbitrary k-ary distributions is n = Θ ( k α2 + k αε ) ( e.g. , see [ 3 , Theorem 13 ] ) . This bound is the sum of two terms : the sample complexity of nonprivate learning ( Θ ( k α2 ) ) plus a term to account for the privacy constraint ( Θ ( k αε ) ) . One interpretation of our result that private sampling requires n = Θ ( k αε ) observations is that the extra privacy term in the complexity of learning can be explained by the complexity of privately generating a single sample with approximately correct distribution . Another implication of our results is that , in some settings , the distributions that are hardest for learning—nonprivate or private—are the easiest for sampling , and vice versa . Consider the simple case of Bernoulli distributions ( i.e. , product distributions with d = 1 ) . The “ hard ” instances for nonprivate learning to within error α are distributions with bias p = 1±α2 , but private sampling is easiest in that parameter regime . In contrast , the “ hard ” instances in our Ω ( 1αε ) lower bound for Bernoulli distributions have bias 10α , that is , close to 0 as opposed to close to 1/2 . A simple variance argument shows that nonprivate learning is easy in that parameter regime , requiring only O ( 1α ) observations . Similarly , for product distributions , we show that the complexity of private sampling is only Θ̃ ( √ d ) when biases are bounded away from 0 and 1 . For the same class , however , the complexity of private and nonprivate learning is Θ ( d ) . Our final point is that our lower bounds for k-ary distributions and general product distributions only require that the sampler generate a value in the support of the distribution with high probability . They thus apply to a weaker problem , similar in spirit to the interior point problem that forms the basis of lower bounds for private learning of thresholds [ 9 ] . Taken together , our results show that studying the complexity of generating a single sample helps us understand the true source of difficulty for certain tasks and sheds light on when we might be able to engage in nontrivial statistical tasks with very little data . Limitations of our results and open questions Our work raises many questions about the complexity of private sampling . First , our upper bounds achieve only the minimal goal of generating a single observation . In most settings , one would presumably want to generate several such samples . One can do so by repeating our algorithms several times on disjoint subsamples , but in general this is not the best way to proceed . Second , we study only three classes of distributions . It is likely that the picture of what is possible for many classes is more complex and nuanced . It would be interesting , for example , to study private sampling for Gaussian distributions , since they demonstrate intriguing data/accuracy tradeoffs for nonprivate sampling [ 4 ] . Societal impact We study the feasibility of basic inference under privacy constraints . Our work is motivated by societal concerns , but focused on fundamental theoretical limits . We do not anticipate direct practical impact .

Summary: This paper initiates an algorithmic study of how to make sampling from discrete distributions private. The formulation is as follows: Given $n$ independent samples from a discrete distribution $P$ over a universe $U$,  output one sample $y$ whose distribution is close to $P$ in total variation distance.  The efficiency goal is to make the number of samples $n$ as small as possible. This appears to be a trivial task - just output any one sample from the input. However, if we require the algorithm to be differentially private then the task is non-trivial and this is the concern of the paper. The paper first considers the problem of private sampling for general distributions over a domain of size $k$ and establishes matching upper and lower bounds on the sample complexity. Then it considers private sampling from product of Bernoulli distributions and establishes upper and lower bounds. In this case, the complexity is not yet completely clear as there is a gap in their bounds.