SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Independent Researcher*
falsifian@falsifian.org
Milind Shyani
mshyani@amazon.com
Nina Mishra
nmishra@amazon.com
How can one publish a dataset with sensitive attributes in a way that both preserves privacy and enables joins with other datasets on those same sensitive attributes? This problem arises in many contexts, e.g., a hospital and an airline may want to jointly determine whether people who take long-haul flights are more likely to catch respiratory infections. If they join their data by a common keyed user identifier such as email address, they can determine the answer, though it breaks privacy. This paper shows how the hospital can generate a private sketch and how the airline can privately join with the hospital's sketch by email address. The proposed solution satisfies pure differential privacy and gives approximate answers to linear queries and optimization problems over those joins. Whereas prior work such as secure function evaluation requires sender/receiver interaction, a distinguishing characteristic of the proposed approach is that it is non-interactive. Consequently, the sketch can be published to a repository for any organization to join with, facilitating data discovery. The accuracy of the method is demonstrated through both theoretical analysis and extensive empirical evidence.
Given a sensitive dataset keyed by individual identities (which must never be revealed), how can we publish a private version of it that can nonetheless be joined by identity to another dataset to produce approximate answers to questions about the join? Can a sender publish this private version to a repository so that receivers can both approximate joint distributions and train ML models, without any interaction between the sender and the receiver?
There are inter-organizational scenarios where answering these questions is valuable. For example, a healthcare provider may have clinical records keyed by patient email address and a private column such as "has cancer". A company wants to determine if employees who work near toxic waste sites have a higher incidence of cancer. We show how two companies can privately join their data in order to build such predictive models. With the rise of data marketplaces, it is more common for organizations to seek to "decorate" their data by adding new columns from other organizations, while preserving privacy. Intra-organizational scenarios also abound. Within a large organization, it is common that data collected in one team is hidden from other teams. With a central repository, not only is data discovery possible, but more private joins can be performed to assess predictive power.
The Repository Setting. A stylized view of the problem is shown in Fig. 1. One party, the sender S, holds a dataset  $D_{S}$  with two columns: a unique identifier to join on, and a value from a small finite domain (in this case, has cancer encoded as  $\pm 1$ ). S would like to share  $D_{S}$  in a privacy-preserving way with many other organizations without explicitly interacting with each one. To this end, S uploads a private representation of  $D_{S}$  to a public repository. Party R is one of many possible receivers, and holds a dataset which has a column with unique identities to join on, and any number of other columns, e.g., latitude and longitude of where their employees work. R's goal is to perform a joint
*Work done while employed by Amazon.
Figure 1: A stylized illustration of the Repository Problem. The sender S uploads a private count sketch capturing people who do and do not have cancer. The receiver R uses the sketch to decorate their data (work location) with a noisy version of S's cancer column. Two noisy columns are generated: one for cancer (+1) and another for not (-1). R can then build an ML model to predict if employees who work near a toxic waste site are more likely to develop cancer.
computation - red box in Fig. 1. It involves the inner join that would normally include only people in both datasets. But, for privacy, the receiver R does not know about the existence of Alice; and even though David is not in S's data, R still receives a label, which is noise. This paper studies how R can learn functions over the inner join of R and S, e.g., does work location predict cancer positivity.
Problem statement Joint computation of linear queries and optimization are described in Subproblems 1.2 and 1.3. The dataset representations should be compatible with both problems simultaneously.
Problem 1.1. Given a dataset  $D_S$  with a unique identity column and one value column with a small finite domain, publish to a repository a single differentially private representation of  $D_S$  which can be combined with any other dataset  $D_R$  (so long as  $D_R$  has a column of unique identities for joining) to solve Subproblems 1.2 and 1.3.
A linear query involves applying some function  $f$  to each element in the join and outputting the sum.
Subproblem 1.2 (Linear Queries). Let  $f$  be a real-valued function. Define
$$
\operatorname {S u m O v e r J o i n} _ {D _ {R}, D _ {S}} (f) = \sum_ {\boldsymbol {i d} \in \operatorname {i d s} \left(D _ {R}\right) \cap \operatorname {i d s} \left(D _ {S}\right)} f \left(D _ {R} [ \boldsymbol {i d} ], D _ {S} [ \boldsymbol {i d} ]\right)
$$
where  $\mathrm{ids}(D)$  is the set of identities that appear in  $D$  and  $D[\mathfrak{id}]$  is the value associated with  $\mathfrak{id}$  in  $D$ . Given access to  $D_R$  and a differentially private representation of  $D_S$ , find an estimate  $\hat{\ell}$  so that
$$
\Pr \left[ \left| \hat {\ell} - \operatorname {S u m O v e r J o i n} _ {D _ {R}, D _ {S}} (f) \right| > \mu \right] <   \delta
$$
In an optimization query, the goal is to find a function  $f$  in a class of functions  $\mathcal{F}$  that approximately minimizes  $\mathrm{SumOverJoin}_{D_R, D_S}(f)$ . Supervised learning with a loss function is a special case.
Subproblem 1.3 (Optimization Queries). Given a class  $\mathcal{F}$  of real-valued functions, let  $f_{\mathrm{opt}} = \arg \min_{f\in \mathcal{F}}\operatorname {SumOverJoin}_{D_R,D_S}(f)$ . Given access to  $D_R$  and a differentially private representation of  $D_S$ , find a function  $\hat{f}$  such that
$$
\Pr \left[ \operatorname {S u m O v e r J o i n} _ {D _ {R}, D _ {S}} (\hat {f}) - \operatorname {S u m O v e r J o i n} _ {D _ {R}, D _ {S}} \left(f _ {\text {o p t}}\right) > \mu \right] <   \delta
$$
Contributions. Problem 1.1 is solved by building upon work on private count sketches. This paper makes the following contributions. (1) Method. We show how to apply the private count sketch to enable a sender to share their data privately, enabling a receiver to approximately join it with their own dataset on individual identifiers to estimate linear queries and learn functions. The identities of the individuals in the sender's data are not revealed. A distinguishing characteristic of the method is that the algorithms are non-interactive, and hence are repository ready. (2) Analysis. A non-trivial mathematical analysis in the form of theorems bounding the error. For linear queries, Problem 1.2,
our main result is Thm. 5.1. For optimization queries, Problem 1.3, our main result is Thm. 6.4, proved using a careful combination of tail bounds. (3) Experiments. (a) We conduct three sets of experiments on public data to compare our solutions to exact answers obtained by joining the data directly, with no privacy. With a reasonable privacy parameter  $(\varepsilon = 1)$  we achieve over  $92\%$  accuracy on the EMNIST dataset, and applying logistic regression on the UCI Adult dataset, we lose less than  $1\%$  accuracy compared to training on the original data with no privacy. (b) We compare to related methods. We adapt the protocol of Zhao et al. [39] to allow for joins and show that our approach is more accurate. We also compare with the LabelDP [21] method on an already-joined dataset, and show that our method performs competitively despite having the harder task of performing a private non-interactive join.
Privacy. Differential privacy [19, 18] is a widely used definition that intuitively guarantees that the outcome of a mechanism is essentially unchanged whether an individual is or is not in the dataset. One of the most commonly studied models of privacy in the literature is the curator model where a trusted centralized repository holds all personal information. The role of the curator is to answer questions about the data in a differentially private way. In the function release problem, the curator must release a single piece of data which can be used to answer an unlimited number of queries [24, 2, 14].
In our case, party S plays the role of curator. The private representation they upload to the public repository can be viewed as an example of function release. It does not depend on the structure of R's dataset or the particular function R aims to compute — S has no knowledge of the parties R who might later download it. Therefore, we can view the sketch as a function accepting as “query” an entire dataset  $D_R$  together with the function to apply to the join.
Private Sketches. Sketching is a common technique used to summarize a dataset. The technique most relevant to this paper is the Count Sketch [10] which we describe in more detail below. Other influential sketching techniques include count min [16], tug of war [4], locality sensitive hashing [25] and AMS [3]. Private variants of sketching have also been considered wherein, for example, the usual sketch is computed and noise is added. Two examples include the private count sketch [6, 39] and the private count min sketch [39]. Count sketch has been used to solve a wide variety of problems, e.g., estimate histograms, frequencies, quantiles or heavy hitters. We add to this list of uses the ability to support data joins. Other examples of private sketch work include the Johnson-Lindenstrauss transform [7] and the loglog sketch [12] — neither require additional noise for privacy. Private locality sensitive hashing [15] has been considered in the context of private function release.
We are not aware of past work that uses the count sketch to privately join. Nevertheless, it is possible to adapt prior private sketching work to our framework. However, since these solutions were not designed with our application in mind, they do not perform as well - see the experiments in §7.2.
Local DP. [6] describes an approach in the local model, where each individual publishes their own noisy data. If individual identities are discarded it precludes joins. If the central server in the local model records individual identities, the identities cannot be published without compromising privacy. While our techniques are similar to past work in local differential privacy, there are important differences: we apply them in the repository setting rather than a local DP setting (see Appendix H for a comparison), the frequencies sketched are always 0 or 1 because unique identifiers are being sketched, and our technique enables joins on those private identifiers. We optimize our use of count sketches for this setting (especially in §6) and provide a novel experimental and theoretical analysis.
Vertically Partitioned Data. [17] study a scenario where a user interactively queries multiple independent databases holding vertical partitions of a dataset. Other results include [30, 31, 33, 36]. Note that in our case, the protocol is non-interactive; each database has an identity column which must be used to join the tables; the identities are private; and some identities may be present in one database and not the other.
Secure Function Evaluation. In the cryptography community, there is extensive work showing how two parties  $A, B$  holding data  $x_{A}, x_{B}$  can jointly compute  $f(x_{A}, x_{B})$  without leaking any information beyond  $f(x_{A}, x_{B})$  [37, 23]. For interactive, multi-party computation, cryptographic techniques are a superior solution. However, this paper is on non-interactive solutions for the repository setting. More comparison can be found in Appendix J.
Notation  $\mathbf{Z}$ ,  $\mathbf{N}$  and  $\mathbf{R}$  denote the sets of integers, nonnegative integers and reals. We let  $\mathcal{U}$  be some set of unique identifiers for people — e.g. strings representing name and address. For datasets  $D$ ,  $D'$ , the join  $D \bowtie D'$  is defined in Definition 3.1, and  $|D| \in \mathbf{N}$  is the number of rows in  $D$ .
Datasets A dataset  $D \subseteq \mathcal{U} \times X$  is a set of pairs (id ∈  $\mathcal{U}, x \in X$ ); we call  $X$  its value domain and its elements rows. In our problem there are two datasets: the sender's  $(D_S)$  and the receiver's  $(D_R)$ . The value domain of  $D_S$  will always be  $X = \{1..k\}$  for some small  $k \in \mathbf{N}$ , but there is no restriction on the value domain of  $D_R$ :  $X$  could be numbers, text, images, etc. We do not allow any id ∈  $\mathcal{U}$  to appear more than once in the same dataset, so in particular there are no duplicate rows.
Joins When the same identity appears in two datasets, we want to combine the associated values.
Definition 3.1. The join of  $D \subseteq \mathcal{U} \times X$  and  $D' \subseteq \mathcal{U} \times Y$  is a dataset  $D \bowtie D' \subseteq \mathcal{U} \times X \times Y$  defined by
$$
(\mathrm {i d}, x, y) \in D \bowtie D ^ {\prime} \iff (\mathrm {i d}, x) \in D \wedge (\mathrm {i d}, y) \in D ^ {\prime}
$$
This is an example of a natural join [13], using the identity column  $D$  and  $D'$  have in common. Given a function  $f: X \times Y \to \mathbf{R}$ , we define the sum over the join:
Definition 3.2.
$$
\operatorname {S u m O v e r J o i n} _ {D, D ^ {\prime}} (f) = \sum_ {(\mathrm {i d}, x, y) \in D \boxtimes D ^ {\prime}} f (x, y)
$$
(This is equivalent to the definition in Subproblem 1.2, but written in terms of  $D \bowtie D'$ .)
Differential Privacy Let  $\varepsilon$  and  $\delta$  be positive real numbers and  $M$  be a randomized algorithm. We say that  $M$  is  $(\varepsilon, \delta)$ -differentially private [19] if for all  $S \subseteq \operatorname{Range}(M)$  and for all pairs of datasets  $D_{1}$  and  $D_{2}$  that differ in the addition or removal of one row,  $\operatorname*{Pr}(M(D_1) \in S) \leq e^{\varepsilon} \operatorname*{Pr}(M(D_2) \in S) + \delta$ , where the probability is taken over the coin flips of  $M$ . In this work we will always set  $\delta = 0$ ; in this case we say  $M$  is  $\varepsilon$ -differentially private; this is called pure differential privacy.
The two-sided geometric distribution The noise we add to ensure privacy will follow the two-sided geometric distribution [22] (see notes in Appendix B.1):
Definition 3.3. The two-sided geometric distribution with parameter  $\alpha$ , denoted  $\mathrm{TGeom}(\alpha)$ , is a distribution over  $\mathbf{Z}$  with probability mass function  $\operatorname*{Pr}[Z = z] = \frac{1 - \alpha}{1 + \alpha}\alpha^{|z|}$ .
Private Count Sketches We use a simplified version of the count sketch [10] as a way to encode a dataset  $D \subseteq \mathcal{U} \times \{1..k\}$ . It is parameterized by a number of buckets  $b$  and two hash functions  $h: \mathcal{U} \times \{1..k\} \to \{1..b\}, s: \mathcal{U} \times \{1..k\} \to \{-1, +1\}$ .
Definition 3.4. Given  $b, h$  and  $s$  as above, the count sketch of a dataset  $D \subseteq \mathcal{U} \times \{1..k\}$  is a vector  $\operatorname{CountSketch}_{b,h,s}(D) = (c_1, \ldots, c_b) \in \mathbf{Z}^b$ , where  $c_j = \sum_{(\mathrm{id}, y) \in D : h(\mathrm{id}, y) = j} s(\mathrm{id}, y)$ .
Adding noise makes a count sketch differentially private:
Definition 3.5. Given  $D, b, h$  and  $s$  as in Definition 3.4, and a privacy parameter  $\varepsilon > 0$ , the private count sketch of  $D$  is a random vector  $\operatorname{PrivCountSketch}_{b,h,s,\varepsilon}(D) = \operatorname{CountSketch}_{b,h,s}(D) + Z_{\varepsilon} \in \mathbf{Z}^b$  where  $Z_{\varepsilon}$  is a vector of  $b$  independent samples from  $\mathrm{TGeom}(e^{-\varepsilon})$  (Definition 3.3).
Since adding or removing a row from  $D$  causes  $\mathrm{CountSketch}_{b,h,s}(D)$  to move a distance of 1 in the  $\ell_1$  metric, we have [6, 39]:
Proposition 3.6.  $\operatorname{PrivCountSketch}_{b,h,s,\varepsilon}(D)$  is  $\varepsilon$ -differentially private.
It is common to repeat the count sketch with multiple independent hash functions to mitigate error caused by hash collisions when there are fewer buckets than items  $(b < n)$ . This is important when the goal is to recover frequencies of repeated items while storing the sketch in as few bits as possible. In contrast, our frequencies are all 1 or 0 (present or not), and our goal is to estimate aggregates of many such frequencies. We are not trying to compress a stream, so we take  $b \approx n$ , at which point
we find the error added by  $Z_{\varepsilon}$  dominates the error from hash collisions. We do not repeat the count sketch because it would improve error only slightly (if at all), and because it would complicate our analysis. In §7.2 we compare to a private sketching method that uses multiple hash functions and Gaussian noise, and find it performs worse than Definition 3.5 for our purposes.
4 Method Overview
We begin with intuitive motivation for the method and then a provide an overview of the steps S and R take to solve Problem 1.1. In §§5 and 6 we explain R's computation in detail.
4.1 Intuition and alternative approaches
Here is a simplified version of our problem. Suppose parties S and R each have a set of individuals' names  $D_{S}, D_{R} \subseteq \mathcal{U} = \{\mathrm{A..Z}\}^{*}$ . R wishes to estimate the number of names in common:  $|D_{S} \cap D_{R}|$ . The names are sensitive and cannot be shared directly. Instead, S must publish a differentially private representation  $C = \mathrm{Private}(D_{S})$  without consulting R, and then R must compute their estimate based on  $C$  and  $D_{R}$ . This is a special case of Subproblem 1.2. Here are some possible approaches.
Secure function evaluation. In cryptography, this problem is called private set intersection cardinality [20, 26, 32]. Cryptographic solutions are well-studied, but require multiple rounds of interaction. In our repository setting, party S publishes a single value  $\mathrm{Private}(D_S)$  and may not interact further. See Appendix J for a discussion on this comparison.
Send the hashes. One might try choosing a secure hash function  $h: \mathcal{U} \to \{0,1\}^*$  and sending  $\mathrm{Private}(D_S) = \{h(x): x \in D_S\}$ . The problem is that  $\mathbf{R}$  would need to know  $h$  to compute  $|D_R \cap D_S|$ . Knowing  $h$  would let them test individuals' membership in  $D_S$ , compromising privacy.
Dot product of noisy vectors. We can view the sets as vectors  $D_{S}, D_{R} \in \{0,1\}^{\mathcal{U}}$ , with one bit for every possible id ∈ U. Then  $D_{S}^{\mathrm{T}}D_{R} = |D_{S} \cap D_{R}|$ . Party S can make their vector  $\varepsilon$ -differentially private by adding noise: Private  $(D_{S}) = D_{S} + Z_{\varepsilon}$ , where  $Z_{\varepsilon} \in \mathbf{Z}^{\mathcal{U}}$  is a vector of independent samples from TGeom  $(e^{-\varepsilon})$ . Party R can estimate  $|D_{R} \cap D_{S}| \approx \operatorname{Private}(D_{S})^{T}D_{R}$ . The problem is that  $\mathcal{U}$  is too big — if identities are  $r$ -bit strings, then Private  $(D_{S})$  takes more than  $2^{r}$  bits to store.
Solution: Private count sketch. The solution we use is a modification of the previous one: S sends a vector with noise added, but first reduces the dimension using a private count sketch.
4.2 Protocol
Our solution to Problem 1.1 has two steps: (1) S publishes a sketch and some parameters (Algorithm 1; figure in Appendix A); (2) R downloads the sketch and runs Algorithm 2 (Subproblem 1.2, linear queries) or Algorithm 3 (Subproblem 1.3, choosing the optimal function from a class).
Although the elements of  $D_S$  are pairs  $(\mathrm{id},y)\in \mathcal{U}\times \{1..k\}$ , we treat them as opaque values for the purpose of computing  $\operatorname{PrivCountSketch}_{b,h,s,\varepsilon}(D_S)$  — so the domain of the hash functions  $h, s$  is  $\mathcal{U}\times \{1..k\}$ . The time complexity of  $\operatorname{Sender}$  is  $O(|D_S| + b)$ , assuming that sampling from  $\operatorname{TGeom}(\alpha)$  and computing the hash functions  $h$  and  $s$  all take constant time.
Algorithm 1 Sender
Let  $\{1..k\}$  be the range of  $D_{S}$ 's value column, i.e.  $D_{S} \subseteq \mathcal{U} \times \{1..k\}$ .
1: Choose the privacy budget  $\varepsilon$  and the number of hash buckets  $b$ .
2:  $h, s \gets \text{ChooseHashFunctions}(b)$
3:  $C\gets \mathrm{PrivCountSketch}_{b,h,s,\varepsilon}(D_S)$
4: Publish  $C \in \mathbf{Z}^b$  and the parameters  $k, b, h, s, \varepsilon$  to a repository.
Choosing hash functions Building a sketch requires first choosing hash functions  $h, s$  with an algorithm ChooseHashFunctions(b). This choice has no effect on privacy: even if  $\forall x, h(x) = s(x) = 1$ , the private count sketch is differentially private. However, higher-quality hash functions will give more accurate results. Thm. 6.4 assumes  $h$  and  $s$  are sampled uniformly from the set of all functions; in practice, cryptographically secure hash functions should give the same guarantees.
5 Linear queries
R can use Algorithm 2 to estimate  $\mathrm{SumOverJoin}_{D_R,D_S}(f)$ , given access to their own dataset  $D_R$  along with  $C = \mathrm{PrivCountSketch}_{b,h,s,\varepsilon}(D_S)$  and the associated parameters  $k,b,h,s$  sent by S. Its time complexity is  $O(k|D_R| + b)$ , assuming  $f(x,y)$  can be computed in constant time. The computation  $Q^{\mathsf{T}}C$  on lines 2-4 can be explained as follows. For every  $(\mathrm{id},x)\in D_R$  and  $y\in \{1..k\}$ , R multiplies  $q(\mathrm{id},x,y) = s(\mathrm{id},y)\cdot f(x,y)$  by the entry at index  $h(\mathrm{id},y)$  in S's sketch. In expectation this equals  $f(x,y)$  if  $(\mathrm{id},y)\in D_S$  and 0 otherwise. The sum of these values over  $(\mathrm{id},x)\in D_R, y\in \{1..k\}$  is  $Q^{\mathsf{T}}C$ , and equals  $\mathrm{SumOverJoin}_{D_R,D_S}(f)$  in expectation. Here the expectation is taken over the output of ChooseHashFunctions and the TGeom  $(\alpha)$  noise.
Algorithm 2 ReceiverLinearQueries
1: Download  $C = \operatorname{PrivCountSketch}_{b,h,s,\varepsilon}(D_S)$  and the parameters  $k, b, h, s$  from the repository.
2: Set  $q(\mathrm{id},x,y) \in \mathbf{R}^b$  to be  $s(\mathrm{id},y)f(x,y)$  at entry  $h(\mathrm{id},y)$  and 0 everywhere else.
3: Construct  $Q = \sum_{(\mathrm{id}, x) \in D_R} \sum_{y=1}^k q(\mathrm{id}, x, y)$ , the query vector.
4: Return  $Q^{\mathsf{T}}C$ .
Theorem 5.1. For any datasets  $D_R \subseteq \mathcal{U} \times X, D_S \subseteq \mathcal{U} \times \{1..k\}$ , function  $f: X \times \{1..K\} \to \mathbf{R}$ , privacy parameter  $\varepsilon > 0$ , number of hash buckets  $b$ , and accuracy parameter  $\mu > 0$ , if  $h$  and  $s$  are drawn from a mutually 4-way independent hash family, the following holds. Let  $M$  be an upper bound on  $|f(x,y)|$ ,  $k$  be the range of the value column of  $D_S$ ,  $n = |D_R \bowtie D_S|$ ,  $C = \mathrm{PrivCountSketch}_{b,h,s,\varepsilon}(D_S)$ . Then
$$
\Pr \left[ \frac {\left| Q ^ {\mathsf {T}} C - \operatorname {S u m O v e r J o i n} _ {D _ {R} , D _ {S}} (f) \right|}{n} > \mu \right] \leq \frac {M ^ {2} \left| D _ {R} \right| (k + 1)}{\mu^ {2} n ^ {2}} \left(\frac {2 e ^ {- \varepsilon}}{(1 - e ^ {- \varepsilon}) ^ {2}} + \frac {\left| D _ {S} \right|}{b ^ {2}}\right)
$$
The probability is taken over  $h, s$  and the noise added to  $\mathrm{PrivCountSketch}_{b,h,s,\varepsilon}(D_S)$ .
We defer the proof to Appendix C. According to the theorem, the size of the join should be sufficiently large for it to be useful.
6 Optimization
Suppose R's dataset has an id column and several feature columns, and S's has an id column and a label column. In this section we show how R can train a model to predict S's labels from R's features. More generally, if R has a class of functions  $\mathcal{F}$  which map  $X\times Y\to \mathbf{R}$ , we show how R can select  $\hat{f}\in \mathcal{F}$  which is close to the optimal function  $f_{\mathrm{opt}}$