Differentially Private Linear Sketches: Efficient Implementations and Applications
Anonymous Author(s)
AffiliationAddress email
Abstract
Linear sketches have been widely adopted to process fast data streams, and theycan be used to accurately answer frequency estimation, approximate top K items,and summarize data distributions.
When data are sensitive, it is desirable toprovide privacy guarantees for linear sketches to preserve private information whiledelivering useful results with theoretical bounds.
We show that linear sketchescan ensure privacy and maintain their unique properties with a small amount ofnoise added at initialization.
From the differentially private linear sketches, weshowcase that the state-of-the-art quantile sketch in the turnstile model can also beprivate and maintain high performance.
Experiments further demonstrate that ourproposed differentially private sketches are quantitatively and qualitatively similarto noise-free sketches with high utilization on synthetic and real datasets
Introduction
Data sketches are fundamental tools for data analysis, statistics, and machine learning [Cormode and
Yi, 2020].
Two of the most widely studied problems in data summaries are frequency estimation andquantile approximation.
Many real world applications need to estimate the frequency of each itemin the database and understand the overall distribution of the database.
These applications includestream processing [Das et al., 2009, Bailis et al., 2017], database management [Misra and Gries, 1982,
Metwally et al., 2005, Zhao et al., 2021a], caching [Zakhary et al., 2020], network monitoring [Guptaet al., 2016, Ivkin et al., 2019], federated learning [Rothchild et al., 2020], among others.
On one hand, the motivation for data sketch algorithms is to efficiently process a large databaseand extract useful knowledge, since computing the exact information for a large amount of data isboth time and memory intensive.
For instance, Munro and Paterson [1980] proved that to find thetrue median of a database with n items using p sequential passes requires at least Ω( n 1 /p ) memory.
On the other hand, to protect user-level privacy, privacy-preserving algorithms limit the disclosureof private information in the database so that an observer cannot infer much about an individual.
Recent works have shown that data sketches can be integrated with privacy-enhancing technologiesto provide insightful information and preserve individual privacy at the same time [Cormode, 2022].
Differential privacy [Dwork et al., 2006] is a widely-accepted definition of privacy.
Recently, researchers have observed that some data sketches are inherently differentially private [Blocki et al.
,2012, Smith et al., 2020], while many other data sketches need modifications to the algorithm to bedifferentially private.
In particular, a substantial amount of literature has focused on differentiallyprivate data sketches for tasks such as linear algebra [Upadhyay, 2014, Arora et al., 2018], cardi- nality estimation [Mir et al., 2011, Pagh and Stausholm, 2020, Dickens et al., 2022] and quantileapproximation [Tzamos et al., 2020, Gillenwater et al., 2021, Alabi et al., 2022].
In this paper, we introduce new differentially private algorithms that support both insertions anddeletions for frequency and top k estimation and quantile approximation.
While many data sketchesassume an insertion-only model [Greenwald and Khanna, 2001, Shrivastava et al., 2004, Karninet al., 2016] or a bounded-deletion model [Jayaram and Woodruff, 2018, Zhao et al., 2021a,b], ouralgorithms build on top of linear sketches [Charikar et al., 2002, Cormode and Muthukrishnan, 2005]and operate in the most general turnstile model, which allows an arbitrary number of insertionsand deletions into the database.
Earlier, researchers attempted to prove Count-Median [Charikaret al., 2002] itself preserves differential privacy, but the authors acknowledged that there are issuesin the proof [Li et al., 2019].
Instead of proving that linear sketches, i.e., both Count-Median and Count-Min, are inherently differentially private, we add a small amount of Gaussian noise at theirinitialization to provide a privacy guarantee while maintaining linear sketches’ original properties,providing high utility for frequency and top K estimations, and keeping update and query algorithmsunchanged.
In addition, we propose the first differentially private quantile sketch in the turnstilemodel by leveraging the differentially private linear sketch.
Our differentially private sketches canbe queried an arbitrary number of times without affecting privacy guarantees based on the postprocessing immunity.
Moreover, in our work, we removed the assumption on the existence of secretuniform random hash functions to provide randomness in the algorithm.
Following prior works [Choiet al., 2020, Smith et al., 2020], we assume ideal random hash functions exist, and this assumptioncan be replaced in practice by cryptographic hash functions [Dickens et al., 2022].
Preliminaries
Consider a database X
= { i t } t ∈
[ N ] of N items that are drawn from a large universe of size U , suchas IPv4 address of size 2 32 , and for each insert or delete operations, one item can be inserted intoor deleted from the database X .
To support ordered statistic such as quantile, we assume that theuniverse is some finite totally ordered data universe.
Definition 2.1.
Given a database X , the frequency of an item x is f ( x ) = (cid:80)
Nt =1 π ( i t = x ) where πreturns 1 if i t is x , and 0 otherwise.
Definition 2.2.
Given a database X of items drawn from an ordered universe, the rank of an item xis R ( x ) =
(cid:80)
Nt =1 π ( i t ≤ x ) where π returns 1 if i t is less or equal to x and 0 otherwise.
Given the large size of N , calculating the actual statistics, such as frequency and quantile, is oftenhard, and hence most applications are satisfied with an approximation .
The randomized frequencyestimation problem takes an accuracy parameter γ and a failure probability β such that, for any itemx , | (cid:98) f
( x ) − f ( x ) | ≤ γ · N with high probability 1 − β , where (cid:98) f ( x ) is the estimated frequency and f ( x )is the true frequency [Cormode and Hadjieleftheriou, 2008].
In addition, the randomized quantileapproximation problem also takes an accuracy parameter γ and a failure probability β
such that, forany item x , | (cid:98) R ( x ) − R ( x ) | ≤ γ · N with high probability 1 − β where (cid:98) R ( x ) is the estimated rankand R ( x ) is the actual rank [Karnin et al., 2016].
2.1 Differential Privacy
Definition 2.3.
Databases X and X ′ are neighbors ( X ∼ X ′ ), if they differ in at most one element.
Through this paper, we assume the update/replace definition of differential privacy instead ofadd/remove definition of differential privacy, in which one item in X is updated or replaced byanother item in X ′
[Vadhan, 2017].
Definition 2.4 (Differential Privacy [Dwork et al., 2006]) .
A randomized algorithm M satisfies( ϵ, δ ) -differential
privacy
( ϵ, δ ) -DP)
if for all neighboring databases X , X ′ and for all possibleevents E in the output range of M , we have
[Equation]
When δ = 0 , ϵ -DP is known as pure DP, and when δ > 0 , ( ϵ , δ )-DP is known as approximate DP.
Definition 2.5 (Gaussian Mechanism [Dwork et al., 2006]) .
Define the ℓ 2 sensitivity of a functionf : N X
(cid:55)→ R d as
[Equation]
The Gaussian mechanism M with noise level σ is then given by
[Equation]
Specifically, the Gaussian mechanism is known to satisfy a stronger notion of privacy known as zeroconcentrated differential privacy (zCDP, defined below); zCDP lies between pure and approximate
DP and can be translated into standard DP notations, as shown in Lemma 2.9.
Moreover, zCDPsatisfies cleaner composition theorems, as shown in Lemma 2.7.
Definition 2.6 (zCDP
[Dwork and Rothblum, 2016, Bun and Steinke, 2016]) .
A randomized mechanism M satisfies ρ -Zero-Concentrated Differential Privacy ( ρ -zCDP), if for all neighboring databases
X, X ′ and all α ∈ (1 , ∞ ) ,
[Equation]
where D α is the Renyi divergence [Van Erven and Harremos, 2014].
Lemma 2.7 (Adaptive composition and Post Processing of zCDP
[Bun and Steinke, 2016]) .
Let
M : X n → Y and M ′ :
X n × Y → Z .
Suppose M satisfies ρ -zCDP and M ′ satisfies ρ ′
-zCDP(as a function of its first argument).
Define M ′′ : X n → Z by M ′′ ( x ) =
M ′ ( x, M ( x )) .
Then M ′′satisfies ( ρ + ρ ′ )
-zCDP.
Lemma 2.8 (Privacy Guarantee of Gaussian mechanism [Dwork et al., 2014, Bun and Steinke,2016]) .
Let f : N X (cid:55)→ R d be an arbitrary d-dimensional function with ℓ 2 sensitivity ∆ 2 =sup neighboring X,X ′ ∥
f ( X ) − f ( X ′ )
∥ 2 .
Then for any ρ > 0 , Gaussian Mechanism with parameterσ 2 = ∆ 22 2 ρ satisfies ρ -zCDP.
Lemma 2.9 (Converting zCDP to DP [Bun and Steinke, 2016]) .
If M satisfies ρ -zCDP then Msatisfies ( ρ + 2 (cid:112) ρ log(1 /δ ) , δ ) -DP.
As we use exclusively Gaussian mechanisms and their composition in our proposed algorithms,our method actually satisfies the stronger ( ϵ, δ ) -DP guarantees than what is implied by zCDP viatechniques from [Balle and Wang, 2018, Dong et al., 2019], which reduces the ϵ parameter by asizable fraction in typical parameter regimes.
We stick to zCDP for clarity and generality, because allour results would apply without changes if we modify the noise into other mechanisms satisfyingzCDP, e.g., the Discrete Gaussian Mechanism [Canonne et al., 2020].
2.2 Revisiting Linear Sketches
Charikar et al.
[2002] proposed the Count-Median sketch, a randomized algorithm that summarizesa database and solves the frequency estimation problem.
The Count-Median sketch uses a d × w arrayof counters, i.e, C[d, w], where all the counters are initialized to zero , and has two sets of independenthash functions h and g .
For each row r , the hash function h r maps input items uniformly onto{ 1 , . . .
, w } and the hash function g r maps input items uniformly onto {− 1 , +1 } .
For item x withvalue v ∈ {− 1 , +1 } , Count-Median sketch updates d counters, one per each row, based on the hashvalues such that for a particular row r ,
g r
( x ) will be added or subtracted to the counter at the h r ( x ) thindex depending on whether x is being inserted or deleted respectively, as shown in Algorithm 1.
Hence, the update time is O ( d ) .
To estimate the frequency of item x , Count-Median sketch will outputthe median 1 ≤ r ≤ d g r ( x ) · C [ r, h r ( x )] , as shown in Algorithm 2.
By updating each row’s counterbased on the hashed value of either 1 or − 1 and reporting the median for query, Count-Mediansketch provides an unbiased estimate.
To reduce the failure probability of bad estimations, d is set to
O (log(1 /β ))
and it uses O ( 1 γ log( 1 β )) space to solve the frequency estimation problem.
Cormode and Muthukrishnan [2005] proposed the Count-Min sketch that shares the same initialization, update, and data structure as Count-Median sketch.
Count-Min sketch also uses O ( 1 γ log( 1 β ))space to solve the frequency estimation problem.
A major difference is that Count-Min sketchmakes all hash functions in set g return positive 1.
As a result, to estimate the frequency of item x ,
Count-Min sketch returns min 1 ≤ r ≤ d C [ r, h r ( x )] instead of the median, as shown in Algorithm 2.
Inaddition, it has the nice property of never underestimating item’s frequency.
Since linear sketches canapproximate an item’s frequency accurately, they also solves the top K approximation problem byreturning the K items associated with the highest estimated frequency.
Moreover, Gilbert et al.
[2002] made the connection between frequency and quantiles to proposethe first universe based RSS quantile sketch in the turnstile model. The observation is that thequantile range, from 0 to the item, can be decomposed into at most log U dyadic intervals [Cormodeet al., 2019] and the sum over the estimated frequencies for these intervals gives the estimated rank. Cormode and Muthukrishnan [2005] proposed the Dyadic Count-Min sketch (DCM) which uses Count-Min sketches for estimating the frequencies of each dyadic interval with space complexity
O ( 1 γ log 2 U log log Uγ ) and update time O (log U log log Uγ ) .
Later, Wang et al.
[2013] leveraged theunbiased property of Count-Median sketches and proposed the Dyadic Count-Median sketch (DCS)which replaces the Count-Min sketch with the Count-Median Sketch [Charikar et al., 2002] to furtherimprove the space complexity to O ( 1 γ log 1 . 5 U log 1 . 5 ( log Uγ )) while maintaining the same update time.
DCS and DCM share the same update and query algorithms as shown in Appendix B. DCM uses
O ( 1 γ log U log log Uγ ) space for each Count-Min sketch and DCS uses O ( 1 γ log 0 .
5 U log 1 . 5 ( log Uγ )) space for each Count-Median sketch, where both of them use O (log log Uγ ) rows. For more specificdetails, [Cormode and Yi, 2020] provide a comprehensive analysis of linear and quantile sketches.
Private Linear Sketches
In this section, we present new algorithms for differentially private linear sketches.
We highlightthat our Private Count Min/Median only require a different initialization while they share the sameupdate (Algorithm 1) and query (Algorithm 2) with the original Count Min/Median.
Therefore, theimplementation of our algorithms is efficient.
Below we show our private initialization.
[Equation] In Algorithm 3, the set of arrays we use is C which consists of log(2 /β )
arrays with length 1 /γ ,which has the same space complexity as original Count Min/Median.
Recall that two neighboringdatabases X and X ′ differ by at most one item.
Therefore, after updating all the items respectively,for each corresponding array in C ( X ) and C ( X ′ ) , they differ by at most two elements and thedifference is at most 1 .
Then the ℓ 2 -sensitivity of the set of arrays C is bounded by
[Equation]
By applying the Gaussian Mechanism (Definition 2.5), we can add independent Gaussian noises
N (0 , σ 2 ) to each counter in C , where σ = (cid:113) log(2 /β )
ρ .
Due to the privacy guarantee of Gaussian
Mechanism (Lemma 2.8), it satisfies ∆ 22 2 σ 2 = ρ -zCDP.
Define E ( β,
[Equation] , ρ ) = [Equation] , for simplicity, we will use E in Algorithm 3 andthe proof in Appendix A.
The private version of Count Min can be derived by adding independent
Gaussian noises N ( E, σ 2 ) to each counter of C , while the private version of Count Median can bederived by adding independent Gaussian noises N (0 , σ 2 ) to each counter of C .
The private versionof Count Min/Median is derived by combining Algorithm 3, Algorithm 1, and Algorithm 2.
Some explanations.
Note that here the difference in the noises we add to Private Count Min/Medianis because we want to retain the nice properties of Count Min (never underestimating) and Count
Median (unbiased estimate).
3.1 Main results about Private Count Min/Median
We present the privacy guarantee and utility analysis of our Private Count Min/Median below.
Recallthat for each item x , we perform update as in Algorithm 1 and query as in Algorithm 2.
In addition,(cid:98) f ( x ) is the output estimated frequency and f ( x ) is the actual frequency.
We begin with the propertiesof Private Count Min.
Note that all the proofs are deferred to Appendix A.
Theorem 3.1.
Private Count Min satisfies ρ
-zCDP regardless of the number of queries.
Furthermore,for each item x , with probability 1 − β , the output (cid:98) f ( x ) satisfies that
[Equation]
Comparison to Count Min.
Comparing our Theorem 3.1 with the conclusion in [Cormode and
Muthukrishnan, 2005], our Private Count Min preserves the nice property that the output will notunderestimate the frequency with high probability.
Furthermore, within the most popular regimewhere the privacy budget ρ is a constant, the additional error bound due to differential privacy isindependent of the size of database N , therefore it will become negligible as N goes large.
Justification of our Gaussian noise.
Note that with high probability, all the noises we add ( E + σ i,j ,σ i,j ∼ N (0 , σ 2 ) ) will be non-negative.
Therefore, the noise we add and the original error induced by
Count Min will directly sum up and lead to larger error in evaluation.
However, we claim that theadditional E ensures that with high probability, for all item x , the output will not underestimate theactual frequency.
This nice property enables the good performance of our Private Count Min whenused in approximate top k task, as shown in Section 5.
Next, Theorem 3.2 shows the properties of Private Count Median.
Theorem 3.2.
Private Count Median satisfies ρ -zCDP regardless of the number of queries.
Furthermore, for each item x , the output (cid:98) f ( x ) satisfies that E (cid:98) f
( x ) =
f
( x ) and with probability 1 − β ,
[Equation]
Comparison to Count Median.
Comparing our Theorem 3.2 with the conclusion in [Charikar et al.
,2002], our Private Count Median preserves the nice property that the output will be an unbiasedestimate of the frequency.
This property enables our use of Private Count Median in quantileestimation below (Section 4).
Furthermore, within the most popular regime where the privacy budgetρ is a constant, the additional error bound due to differential privacy is independent of the size ofdatabase N , thus it will become negligible as N goes large.
Better performance in evaluation.
Since the noise we add is independent to the error of Count
Median, these two errors may cancel each other out. In addition, the scale of the whole error issmaller than a direct summation of the scale of these two errors, which will lead to a much betterperformance in evaluation (Section 5) compared to the worst case bound.
Private Quantile Sketches
In this section, we apply our Private Count Median to state of the art quantile sketches in the turnstilemodel.
Our private Dyadic Count-Median sketch can estimate all the quantiles accurately at the sametime while ensuring differential privacy.
Revisiting DCS
In [Wang et al., 2013], it is shown that DCS can return all γ -approximate quantiles with constantprobability using space O (cid:16) 1 γ log 1 .
5 U log 1 .
5 ( log Uγ ) (cid:17) .
More specifically, the sketch structure hereconsists of log U Count Medians, each Count Median uses a counter arrays C , which is d × wcounters.
The choice of d, w follows d = Θ (cid:16) log( log Uγ ) (cid:17) and w = O (cid:16)(cid:113) log U log( log Uγ ) /γ
(cid:17) .
4.2 Private DCS
In this work, we aim to estimate the quantiles accurately while preserving privacy.
We do this by replacing Count Median with Private Count Median, which bases on the same structure as Count
Median discussed above.
Given the privacy budget ρ , the privacy budget of each Private Count
Median is thus ρ 0 = ρ log U , due to composition of zCDP (Lemma 2.7).
The ℓ 2 -sensitivity of each
Private Count Median is
[Equation]
To keep the whole algorithm ρ -zCDP, it suffices to keep each Count Median ρ 0zCDP (Lemma 2.7).
Therefore, Gaussian Mechanism (Definition 2.5) with σ=
O (cid:16) log U log (cid:16) log Uγ (cid:17) /ρ
(cid:17) ensures ρ
-zCDP (Lemma 2.8).
Similar to Lemma A.1, define E ( γ, U ) =
O (cid:32)(cid:114)
[Equation]
log [Equation] [Equation] (cid:33) , we can prove that with constant probability, all the
Gaussian noises we add to all log U Count Medians are bounded by E (for simplicity, we use E torepresent E ( γ, U ) ).
Conditioned on the high probability event above, we prove that for a fixed quantile, the estimated quantile will be accurate with high probability.
As has been proven in Theorem 3.2, the output estimated frequency is unbiased for any item.
Therefore, similar to [Wang et al., 2013], for any item x (corresponding to a fixed Count Median), we have the output (cid:98) f ( x ) of that Count Median satisfies
[Equation]
By a union bound, with probability 1 − log U
× O (cid:16) γ log U (cid:17) = 1 − O ( γ ) , for any item correspondingto this fixed quantile, the error of Count Median is bounded by 1 w · N + E .
Conditioned upon thisevent, by Hoeffding’s inequality, with probability 1 − O (cid:16) γ log U (cid:17) , the sum of log U such independenterrors is bounded by
[Equation]
where E ′ =
O
[Equation]
(cid:33) .
To sum up, for a fixed quantile, withprobability 1 − O ( γ ) , the estimating error is bounded by γ · N + E ′ .
Finally, apply another union bound on the 1 γ different quantiles, with constant probability, all thequantiles are estimated accurately (within the error bound (4)).
Note that similar to [Wang et al.
,2013], the failure probability here is a constant.
For any failure probability β , we can further increased by a factor of log 1 β to reduce this failure probability to β .
Take-away of Private DCS.
First, our Private DCS has a same space complexity as the original
DCS.
In addition, according to (4), the additional error bound is proportional to log U log 1 γ √ ρ (ignoringlog log terms), and independent to the size of database N .
In the most popular regime where theprivacy budget ρ is a constant, the additional error bound only appears as lower order terms, whichwill become negligible as N goes large.
Evaluation5.1 Data Sets
The experimental evaluation is conducted using both synthetic and real world data sets.
We consider the synthetic Zipf dataset Zipf [2016] and the real CAIDA Anonymized Internet Trace 2015dataset pas.
For each independent run in the experiments, we use an input database size N = 10 5 .
• Zipf Dataset : Items are drawn from a bounded universe of size 2 16 and items’ frequenciesfollow the Zipf Law
Zipf [2016].• 2015 CAIDA Dataset : The CAIDA dataset is collected from the Chicago high-speedmonitor.
Items are the source IP addresses with bounded universe size of 2 32 .
5.2 Metrics
In all experiments, we average the various metrics over 5 independent runs to minimize the measurement variance as all data sketches have randomness depending on the hash function.
The metricsused in the experiments are:
• Average Relative Error : Let the set Ψ denotes all unique items in the database.
The averagerelative error (ARE) is computed based on Ψ in which 1Ψ (cid:80) e ∈ Ψ | f ( e ) − (cid:98)
f
( e ) | f
( e ) .
• F1 Score : F1 score is the harmonic mean of the precision and recall ( 2 · precision · recall precision + recall ).
• Average Rank Error : For each evenly spaced quantile and its associated item, we averagethe distance between the true rank and estimated
rank. We use ARE to evaluate sketch performance on frequency estimation and F1 score to evaluate thesketch’s performance in identifying the top 10 items.
For quantile approximation, we consider the mevenly spaced quantiles and items.
For instance, if m = 1 , we consider the rank error for the medianitem; if m = 2 , we consider then average rank error for the 33 rd and 67 th percentile items.
Lower
ARE and average rank error, and higher F1 score indicate better approximation.
5.3 Private Linear Sketches Experiments
[Figure] To evaluate the utility of DP linear sketches, we compare the average relative error (ARE) and F1score for frequency estimation and identify the top 10 items, respectively.
As shown in Figure 1, thex-axis represents the space budget for each sketch (from 9.2 KB to 147.3 KB), and the y-axis denotes
ARE or F1 score.
The DP linear sketches use ρ ∈ { 0 . 1 , 1 , 10 } in which lower ρ value indicates morenoise need to be added, and all sketches assume β = 1% .
For frequency estimation, the performance of private Count-Median sketches with various privacybudgets is basically equivalent to the performance of the non-private Count-Median sketch.
Underdifferent space and privacy budgets, they have minimal difference in ARE for both Zipf and CAIDAdatasets, meaning that, while providing strong privacy guarantee, the estimated frequencies are stillvery accurate.
The accurate estimation of private Count-Median is primarily due to the unbiasednature of Count-Median in which, by adding Gaussian noise, the private Count-Median still providesunbiased estimation for an item’s frequency as proved in Theorem 3.2.
As shown in both Figure 1(a)and Figure 1(b), the performance of private Count-Min degrades when the space budget increasesor the privacy budget decreases.
This behavior is expected as the upper bound on the frequency error in Theorem 3.1 has a dependency on both γ and ρ .
In order to preserve the property of notunderestimating an item’s frequency, the private Count-Min sketch needs to add larger noise toeach counter when the number of counters increases.
As a result, the estimated frequencies forlow-frequency items become inflated and this in turn decreases the overall accuracy.
For approximate top 10 items, private Count-Median has similar performance to Count-Median.
Since non-private and private Count-Median are unbiased, they may underestimate the frequency oftrue top K items and decrease the recall.
On the other hand, the property of no underestimation isdesirable for approximate top K items.
In particular, non-private and private Count-Min sketch scorehigh F1 scores for all datasets.
While providing privacy guarantees, private Count-Min achieve 1.
F1 scores for all space and all privacy budgets in Zipf dataset, as shown in Figure 1(c).
5.4 Private Quantile Sketch Experiments
[Figure]
To evaluate the utility of DP DCS, we compare the average rank error.
As shown in Figure 2, thex-axis represents the number of evenly spaced quantiles, and the y-axis denotes the average rank error.
The DP DCS use privacy budget ρ ∈ { 0 . 1 , 1 , 10 } and all sketches assume γ = 1% .
For the quantile approximation, we observe that the increase in the number of evenly spaced quantilesdoes not impact the average rank error, as shown in both Figure 2(a) and Figure 2(b).
Since the
CAIDA dataset universe size ( 2 32 ) is larger than Zipf dataset universe size ( 2 16 ), the average rankerror in the CAIDA dataset is larger than the average rank error in the Zipf dataset.
As shown in
Equation (4), the error bound has a term depending on the universe size in which a large universesize leads to more error.
When the privacy budget decreases, the average rank error increases as morenoise needs to be added.
Comparing DP DCS with strong privacy ( ρ = 0 . 1 ) to DCS, the increase inrank error is relatively small compared to the database size of 10 5 .
In addition, the desired rank errorupper bound is γ · N = 10 3 and all the rank errors are one order of magnitude lower.
Conclusion
In this work, we demonstrate that linear sketches can be made differentially private and provideuseful information while maintaining their original properties by adding a small amount of Gaussiannoise at initialization.
In addition, leveraging the private Count-Median sketch, we propose the DP
DCS for quantile approximation in the turnstile model, and the DP DCS achieves low rank erroreven for a large data universe.
Moreover, for all proposed algorithms, when the privacy budget isconstant, the additional error due to privacy is independent of the database size and the error willbecome negligible when the database grows larger.
As a result, we believe our proposed algorithmsare efficient and practical for real-world systems and enable these systems to perform data analysisand machine learning tasks privately.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] Claims in abstract and introduction reflect ourcontributions.(b)
Did you describe the limitations of your work?
[Yes] In Theorem 3.1, the private CountMin frequency estimation’s additional error has a dependency of (cid:113) log 1 γ in order tokeep the property of not underestimating item’s frequency.
Assume database size isfixed, decreasing gamma will increase the average error.
Note the error is independentfrom the database size, and when database size grows, the error will become negligible.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] Apply
our proposed algorithms to current systems will protect user privacy.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes](b) Did you include complete proofs of all theoretical results?
[Yes] We put some proves
in Appendix A due to space limits
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] We will provide
an url to our Github Repo once the paper is accepted.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] We provide the parameters used for the experiments.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] We didn’t use random seed
(d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)?
[N/A] we didn’t use any external
resources beside a macbook pro.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
We did not useexisting assets.
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
We did not use crowdsourcing or conducted research with human subjects.
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A](
c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]