Original: D IFFERENTIALLY P RIVATE F RACTIONAL F REQUENCY M OMENTS E STIMATION WITH P OLYLOGARITHMIC
Revised: D IFFERENTIALLY P RIVATE F RACTIONAL F REQUENCY M OMENTS E STIMATION WITH P OLYLOGARITHMIC

Original: S PACE
Revised: S PACE

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We prove that F p sketch, a well-celebrated streaming algorithm for frequency moments estimation, is differentially private as is when p ∈ (0 , 1] .
Revised: We prove that F p sketch, a well-celebrated streaming algorithm for frequency moments estimation, is differentially private as is when p ∈ (0 , 1] .

Original: F p sketch uses only polylogarithmic space, exponentially better than existing DP baselines and only worse than the optimal non-private baseline by a logarithmic factor.
Revised: F p sketch uses only polylogarithmic space, exponentially better than existing DP baselines and only worse than the optimal non-private baseline by a logarithmic factor.

Original: The evaluation shows that F p sketch can achieve reasonable accuracy with differential privacy guarantee.
Revised: The evaluation shows that F p sketch can achieve reasonable accuracy with differential privacy guarantee.

Original: The evaluation code is included in the supplementary material.
Revised: The evaluation code is included in the supplementary material.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Counting is one of the most fundamental operations in almost every area of computer science.
Revised: Counting is one of the most fundamental operations in almost every area of computer science.

Original: It typically refers to estimating the cardinality (the 0 th frequency moment) of a given set.
Revised: It typically refers to estimating the cardinality (the 0 th frequency moment) of a given set.

Original: However, counting can actually refer to the process of estimating a broader class of statistics, namely p th frequency moment, denoted F p .
Revised: However, counting can actually refer to the process of estimating a broader class of statistics, namely p th frequency moment, denoted F p .

Original: Frequency moments estimation is at the core of various important statistical problems.
Revised: Frequency moments estimation is at the core of various important statistical problems.

Original: F 1 is used for data mining (Cormode et al., 2005) and hypothesis tests (Indyk & McGregor, 2008).
Revised: F 1 is used for data mining (Cormode et al., 2005) and hypothesis tests (Indyk & McGregor, 2008).

Original: F 2 has applications in calculating Gini index (Lorenz, 1905; Gini, 1912) and surprise index (Good, 1989), training random forests (Breiman, 2001), numerical linear algebra (Clarkson & Woodruff, 2009; Sarlos, 2006) and network anomaly detection (Krishnamurthy et al., 2003; Thorup & Zhang, 2004).
Revised: F 2 has applications in calculating Gini index (Lorenz, 1905; Gini, 1912) and surprise index (Good, 1989), training random forests (Breiman, 2001), numerical linear algebra (Clarkson & Woodruff, 2009; Sarlos, 2006) and network anomaly detection (Krishnamurthy et al., 2003; Thorup & Zhang, 2004).

Original: Fractional frequency moments are used in Shannon entropy estimation (Harvey et al., 2008; Zhao et al., 2007) and image decomposition (Geiger et al., 1999).
Revised: Fractional frequency moments are used in Shannon entropy estimation (Harvey et al., 2008; Zhao et al., 2007) and image decomposition (Geiger et al., 1999).

Original: Non-private frequency moments estimation is systematically studied in the data streaming model (Alon et al., 1999;
Revised: Non-private frequency moments estimation is systematically studied in the data streaming model (Alon et al., 1999;

Original: Charikar et al., 2002; Thorup & Zhang, 2004; Feigenbaum et al., 2002;
Revised: Charikar et al., 2002; Thorup & Zhang, 2004; Feigenbaum et al., 2002;

Original: Indyk, 2006; Li, 2008; Kane et al., 2010; Nelson & Woodruff, 2009; 2010; Kane et al., 2011).
Revised: Indyk, 2006; Li, 2008; Kane et al., 2010; Nelson & Woodruff, 2009; 2010; Kane et al., 2011).

Original: This model assumes extremely limited storage such as network routers.
Revised: This model assumes extremely limited storage such as network routers.

Original: The optimal non-private algorithm (Kane et al., 2011) uses only polylogarithmic space to maintain frequency moments.
Revised: The optimal non-private algorithm (Kane et al., 2011) uses only polylogarithmic space to maintain frequency moments.

Original: In the present work, we inherit the low space complexity requirement for the versatility of the algorithm.
Revised: In the present work, we inherit the low space complexity requirement for the versatility of the algorithm.

Original: The data being counted sometimes contains sensitive information.
Revised: The data being counted sometimes contains sensitive information.

Original: For example, to calculate Gini index, the data should contain pairs of ID and income.
Revised: For example, to calculate Gini index, the data should contain pairs of ID and income.

Original: Frequency moments of such data, if published, might leak sensitive information.
Revised: Frequency moments of such data, if published, might leak sensitive information.

Original: To mitigate, the gold standard of differential privacy (DP) should be applied.
Revised: To mitigate, the gold standard of differential privacy (DP) should be applied.

Original: Special cases of DP frequency moments estimation such as p = 0 , 1 , 2 are well-studied in a wide spectrum of works (Choi et al., 2020; Smith et al., 2020;
Revised: Special cases of DP frequency moments estimation such as p = 0 , 1 , 2 are well-studied in a wide spectrum of works (Choi et al., 2020; Smith et al., 2020;

Original: Blocki et al., 2012; Sheffet, 2017;
Revised: Blocki et al., 2012; Sheffet, 2017;

Original: Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021; Mir et al., 2011).
Revised: Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021; Mir et al., 2011).

Original: In the present work, we make the ﬁrst customized effort towards DP estimation of fractional frequency moments, i.e. p ∈ (0 , 1] with low space complexity.
Revised: In the present work, we make the ﬁrst customized effort towards DP estimation of fractional frequency moments, i.e. p ∈ (0 , 1] with low space complexity.

Original: We show that a well-known streaming algorithm, namely F p sketch (Indyk, 2006), preserves differential privacy as is.
Revised: We show that a well-known streaming algorithm, namely F p sketch (Indyk, 2006), preserves differential privacy as is.

Original: With its small space complexity, F p sketch elegantly solves the trilemma between efﬁciency, accuracy, and privacy.
Revised: With its small space complexity, F p sketch elegantly solves the trilemma between efﬁciency, accuracy, and privacy.

Original: Problem Formulation.
Revised: Problem Formulation.

Original: We use bold lowercase letters to denote vectors ( e.g. a , b , c ) and bold uppercase letters to denote matrices ( e.g. A , B , C ).
Revised: We use bold lowercase letters to denote vectors ( e.g. a , b , c ) and bold uppercase letters to denote matrices ( e.g. A , B , C ).

Original: { 1 , · · · , n } is denoted by [ n ] .
Revised: { 1 , · · · , n } is denoted by [ n ] .

Original: Let S
Revised: Let S

Original: =
Revised: =

Original: ( k 1 , v 1 ) , · · · , ( k n , v n ) } ( n ≥ 1) be a stream of key-value pairs where k i ∈
Revised: ( k 1 , v 1 ) , · · · , ( k n , v n ) } ( n ≥ 1) be a stream of key-value pairs where k i ∈

Original: [ m ] ( m ≥ 2) , v i ∈
Revised: [ m ] ( m ≥ 2) , v i ∈

Original: [ M ] ( M ≥ 1) 1 .
Revised: [ M ] ( M ≥ 1) 1 .

Original: We would like to design a randomized mechanism M that estimates the p th frequency moment:
Revised: We would like to design a randomized mechanism M that estimates the p th frequency moment:

Original: [Equation]
Revised: [Equation]

Original: for p ∈ (0 , 1] where I is an indicator function returning 1 if k = k
Revised: for p ∈ (0 , 1] where I is an indicator function returning 1 if k = k

Original: i
Revised: i

Original: and 0 otherwise.
Revised: and 0 otherwise.

Original: To provide rigorous privacy guarantee, M should preserve differential privacy as deﬁned below.
Revised: To provide rigorous privacy guarantee, M should preserve differential privacy as deﬁned below.

Original: In our setting, neighboring data streams differ in one key-value pair.
Revised: In our setting, neighboring data streams differ in one key-value pair.

Original: Deﬁnition 1 ( ( (cid:15), δ ) -Differential Privacy) .
Revised: Deﬁnition 1 ( ( (cid:15), δ ) -Differential Privacy) .

Original: A randomized algorithm M is said to preserve ( (cid:15), δ ) -DP
Revised: A randomized algorithm M is said to preserve ( (cid:15), δ ) -DP

Original: if for two neighboring datasets S , S (cid:48) and any measurable subset of the output space s ,
Revised: if for two neighboring datasets S , S (cid:48) and any measurable subset of the output space s ,

Original: [Equation]
Revised: [Equation]

Original: When δ = 0 , we omit it and denote the privacy guarantee as (cid:15) -DP.
Revised: When δ = 0 , we omit it and denote the privacy guarantee as (cid:15) -DP.

Original: Oftentimes, n, m is large ( e.g. IP streams on routers)
Revised: Oftentimes, n, m is large ( e.g. IP streams on routers)

Original: so M should take polylogarithmic space in terms of n, m .
Revised: so M should take polylogarithmic space in terms of n, m .

Original: Proof Intuition.
Revised: Proof Intuition.

Original: We summarize the intuition behind the proof that F p sketch is differentially private when p ∈ (0 , 1] .
Revised: We summarize the intuition behind the proof that F p sketch is differentially private when p ∈ (0 , 1] .

Original: Recall that when proving DP for traditional mechanisms such as the Gaussian mechanism, the core is to upper-bound the ratio P ( x ) Q ( x ) where P ( x ) and Q ( x ) are the probability density functions of outputs when the inputs are neighboring datasets.
Revised: Recall that when proving DP for traditional mechanisms such as the Gaussian mechanism, the core is to upper-bound the ratio P ( x ) Q ( x ) where P ( x ) and Q ( x ) are the probability density functions of outputs when the inputs are neighboring datasets.

Original: In the proof of Gaussian mechanism, P ( x ) and Q ( x ) can be viewed as a horizontal translation of each other and the distance between their mean values is the sensitivity of the output.
Revised: In the proof of Gaussian mechanism, P ( x ) and Q ( x ) can be viewed as a horizontal translation of each other and the distance between their mean values is the sensitivity of the output.

Original: For F p , however, neighboring inputs do not translate the output distribution but instead change its scale.
Revised: For F p , however, neighboring inputs do not translate the output distribution but instead change its scale.

Original: For example, when p = 2 , P ( x ) and Q ( x ) are Gaussian distributions with the same mean and different variance.
Revised: For example, when p = 2 , P ( x ) and Q ( x ) are Gaussian distributions with the same mean and different variance.

Original: Inspired by the analogy to Gaussian mechanism, we need to address the below two questions to prove differential privacy for F p sketches.
Revised: Inspired by the analogy to Gaussian mechanism, we need to address the below two questions to prove differential privacy for F p sketches.

Original: Q1.
Revised: Q1.

Original: How to bound the difference between the scales of P ( x ) and Q ( x ) ?
Revised: How to bound the difference between the scales of P ( x ) and Q ( x ) ?

Original: Q2.
Revised: Q2.

Original: How to bound the ratio between the density functions of P ( x ) and Q ( x ) ?
Revised: How to bound the ratio between the density functions of P ( x ) and Q ( x ) ?

Original: To answer Q1, we propose a new sensitivity deﬁnition called pure multiplicative sensitivity .
Revised: To answer Q1, we propose a new sensitivity deﬁnition called pure multiplicative sensitivity .

Original: Pure multiplicative sensitivity depicts the maximal multiplicative change in the output when the inputs are neighboring datasets.
Revised: Pure multiplicative sensitivity depicts the maximal multiplicative change in the output when the inputs are neighboring datasets.

Original: We analyze frequency moments estimation and ﬁnd that its pure multiplicative sensitivity is approximately max { 2 2 p − 2 , 2 2 − 2 p } when p ∈ (0 , 1] and n (cid:29) M .
Revised: We analyze frequency moments estimation and ﬁnd that its pure multiplicative sensitivity is approximately max { 2 2 p − 2 , 2 2 − 2 p } when p ∈ (0 , 1] and n (cid:29) M .

Original: To answer Q2, we ﬁrst revisit the special case of p = 1 .
Revised: To answer Q2, we ﬁrst revisit the special case of p = 1 .

Original: As shown by Mir et al.
Revised: As shown by Mir et al.

Original: (2011), when p = 1 , P ( x )
Revised: (2011), when p = 1 , P ( x )

Original: Q ( x ) is rigorously upper-bounded and thus F 1 sketch preserves (cid:15)
Revised: Q ( x ) is rigorously upper-bounded and thus F 1 sketch preserves (cid:15)

Original: -DP.
Revised: -DP.

Original: By analogy, we conjecture that F p , p ∈ (0 , 1] also satisﬁes similar properties, which is doubly conﬁrmed by the numerically simulated plots in Figure 2.
Revised: By analogy, we conjecture that F p , p ∈ (0 , 1] also satisﬁes similar properties, which is doubly conﬁrmed by the numerically simulated plots in Figure 2.

Original: The conjecture is formally proved in Theorem 3.
Revised: The conjecture is formally proved in Theorem 3.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Frequency moments estimation is thoroughly studied in the data streaming model.
Revised: Frequency moments estimation is thoroughly studied in the data streaming model.

Original: Alon et al.
Revised: Alon et al.

Original: (1999) proposed the ﬁrst space-efﬁcient algorithm for estimating p th frequency moments when p is integer.
Revised: (1999) proposed the ﬁrst space-efﬁcient algorithm for estimating p th frequency moments when p is integer.

Original: Indyk (2006) extended the use case from integer moments to fractional moments using stable distributions.
Revised: Indyk (2006) extended the use case from integer moments to fractional moments using stable distributions.

Original: A line of following works improve Indyk’s algorithm in various aspects such as space complexity (Kane et al., 2010; Nelson & Woodruff, 2009), time complexity (Nelson & Woodruff, 2010; Kane et al., 2011) or accuracy (Li, 2008; 2009).
Revised: A line of following works improve Indyk’s algorithm in various aspects such as space complexity (Kane et al., 2010; Nelson & Woodruff, 2009), time complexity (Nelson & Woodruff, 2010; Kane et al., 2011) or accuracy (Li, 2008; 2009).

Original: Several special cases in private frequency moments estimation such as p = 0 , 1 , 2 were also well studied.
Revised: Several special cases in private frequency moments estimation such as p = 0 , 1 , 2 were also well studied.

Original: Choi et al.
Revised: Choi et al.

Original: (2020) and Smith et al.
Revised: (2020) and Smith et al.

Original: (2020) studied differentially private F 0 estimation.
Revised: (2020) studied differentially private F 0 estimation.

Original: They separately proved that the Flajolet-Martin sketch is differentially private as is.
Revised: They separately proved that the Flajolet-Martin sketch is differentially private as is.

Original: Several independent works (Blocki et al., 2012; Sheffet, 2017; Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021) studied the differential privacy guarantee in the special case p = 2 under the name of Johnson-Lindenstrauss projection.
Revised: Several independent works (Blocki et al., 2012; Sheffet, 2017; Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021) studied the differential privacy guarantee in the special case p = 2 under the name of Johnson-Lindenstrauss projection.

Original: On the other hand, there is barely any prior work focusing on differentially private fractional frequency moments estimation.
Revised: On the other hand, there is barely any prior work focusing on differentially private fractional frequency moments estimation.

Original: Differentially private distribution estimation algorithms (Acs et al., 2012; Xu et al., 2013; Bassily & Smith, 2015; Suresh, 2019; Wang et al., 2019) can be used to provide a differentially private estimation of fractional frequency moments.
Revised: Differentially private distribution estimation algorithms (Acs et al., 2012; Xu et al., 2013; Bassily & Smith, 2015; Suresh, 2019; Wang et al., 2019) can be used to provide a differentially private estimation of fractional frequency moments.

Original: However, they are overkill as their outputs contain much more information than the queried fractional frequency moment.
Revised: However, they are overkill as their outputs contain much more information than the queried fractional frequency moment.

Original: They only provide sub-optimal privacy-utility trade-off and are exponentially worse in terms of space complexity.
Revised: They only provide sub-optimal privacy-utility trade-off and are exponentially worse in terms of space complexity.

Original: Datar et al.
Revised: Datar et al.

Original: (2004) considered a similar (but not the same) mathematical problem to the present work when designing a locality-sensitive hashing scheme.
Revised: (2004) considered a similar (but not the same) mathematical problem to the present work when designing a locality-sensitive hashing scheme.

Original: However, their analysis focuses on the simple cases when p = 1 and p = 2 and totally depends on numerical analysis for p ∈ (0 , 1) .
Revised: However, their analysis focuses on the simple cases when p = 1 and p = 2 and totally depends on numerical analysis for p ∈ (0 , 1) .

Original:  D IFFERENTIALLY P RIVATE F REQUENCY M OMENTS E STIMATION
Revised:  D IFFERENTIALLY P RIVATE F REQUENCY M OMENTS E STIMATION

Original: In this section, we ﬁrst revisit F p sketch and then prove the differential privacy guarantee for F p sketch step by step.
Revised: In this section, we ﬁrst revisit F p sketch and then prove the differential privacy guarantee for F p sketch step by step.

Original: Different from most differential privacy analyses based on additive sensitivity, our proof depends on a variant of the multiplicative sensitivity (Dwork et al., 2015) called pure multiplicative sensitivity .
Revised: Different from most differential privacy analyses based on additive sensitivity, our proof depends on a variant of the multiplicative sensitivity (Dwork et al., 2015) called pure multiplicative sensitivity .

Original: We give the ﬁrst analysis of pure multiplicative sensitivity for p -th frequency moments.
Revised: We give the ﬁrst analysis of pure multiplicative sensitivity for p -th frequency moments.

Original: Then we motivate the differential privacy proof using a special case when p = 1 .
Revised: Then we motivate the differential privacy proof using a special case when p = 1 .

Original: Finally we proceed to the general proof that F p sketch preserves differential privacy.
Revised: Finally we proceed to the general proof that F p sketch preserves differential privacy.

Original: The main challenge stems from the fact that the density functions of p -stable distributions have no close-form expressions when p ∈ (0 , 1) .
Revised: The main challenge stems from the fact that the density functions of p -stable distributions have no close-form expressions when p ∈ (0 , 1) .

Original: 3. R EVISITING F p S KETCH
Revised: 3. R EVISITING F p S KETCH

Original: For completeness, we revisit the well-celebrated F p sketch by Indyk (2006) (also known as stable projection or compressed counting).
Revised: For completeness, we revisit the well-celebrated F p sketch by Indyk (2006) (also known as stable projection or compressed counting).

Original: We ﬁrst introduce p -stable distribution, the basic building block in F p sketch.
Revised: We ﬁrst introduce p -stable distribution, the basic building block in F p sketch.

Original: Then we review how to construct and query F p sketch using stable distributions.
Revised: Then we review how to construct and query F p sketch using stable distributions.

Original: Deﬁnition 2 ( p -stable distribution) .
Revised: Deﬁnition 2 ( p -stable distribution) .

Original: A random variable X follows a β -skewed p -stable distribution if its characteristic function is
Revised: A random variable X follows a β -skewed p -stable distribution if its characteristic function is

Original: [Equation]
Revised: [Equation]

Original: where − 1 ≤ β ≤ 1 is the skewness parameter, ζ > 0 is the scale parameter to the p
Revised: where − 1 ≤ β ≤ 1 is the skewness parameter, ζ > 0 is the scale parameter to the p

Original: th power.
Revised: th power.

Original: In this paper, we focus on stable distributions with β = 0 , namely symmetric stable distributions.
Revised: In this paper, we focus on stable distributions with β = 0 , namely symmetric stable distributions.

Original: We denote a symmetric p -stable distribution by D p,ζ , and slightly abuse the notation to denote the density function as D p,ζ ( x ) .
Revised: We denote a symmetric p -stable distribution by D p,ζ , and slightly abuse the notation to denote the density function as D p,ζ ( x ) .

Original: Note that the density function is the inverse Fourier transform of the characteristic function.
Revised: Note that the density function is the inverse Fourier transform of the characteristic function.

Original: [Equation]
Revised: [Equation]

Original: If two independent random variables X 1 , X 2 ∼ D p, 1 , then C 1 X 1 + C 

Revised: If two independent random variables X 1 , X 2 ∼ D p, 1 , then C 1 X 1 + C

Original: X
Revised: X

Original: 2 ∼ D p,C p 1 + C p 2 .
Revised: 2 ∼ D p,C p 1 + C p 2 .

Original: We refer to this property as p -stability .
Revised: We refer to this property as p -stability .

Original: F p sketch leverages the p -stability of these distributions to keep track of the frequency moments.
Revised: F p sketch leverages the p -stability of these distributions to keep track of the frequency moments.

Original: The pseudo-code for vanilla F p sketch is presented in Algorithm 1.
Revised: The pseudo-code for vanilla F p sketch is presented in Algorithm 1.

Original: To construct, a sketch of size r is initialized to all zeros and a projection matrix P is sampled from D r × m p,(line 2).
Revised: To construct, a sketch of size r is initialized to all zeros and a projection matrix P is sampled from D r × m p,(line 2).

Original: For each incoming key-value pair ( k i , v i ) , we multiply the one-hot encoding of k i scaled by v i with the projection matrix
Revised: For each incoming key-value pair ( k i , v i ) , we multiply the one-hot encoding of k i scaled by v i with the projection matrix

Original: P and add it to the sketch (line 4).
Revised: P and add it to the sketch (line 4).

Original: [Equation]
Revised: [Equation]

Original: To query the sketch, we estimate ζ from a using various estimators such as median, inter-quantile, geometric mean or harmonic mean as suggested by Indyk (2006), Li (2008) and Li (2009).
Revised: To query the sketch, we estimate ζ from a using various estimators such as median, inter-quantile, geometric mean or harmonic mean as suggested by Indyk (2006), Li (2008) and Li (2009).

Original: Algorithm 1:
Revised: Algorithm 1:

Original: 3. P URE MULTIPLICATIVE SENSITIVITY OF FREQUENCY MOMENTS ESTIMATION
Revised: 3. P URE MULTIPLICATIVE SENSITIVITY OF FREQUENCY MOMENTS ESTIMATION

Original: As we will see in the following two subsections, the differential privacy proof for F p sketch depends on the pure multiplicative sensitivity of p -th frequency moments.
Revised: As we will see in the following two subsections, the differential privacy proof for F p sketch depends on the pure multiplicative sensitivity of p -th frequency moments.

Original: As the ﬁrst step, we give the deﬁnition of pure multiplicative differential privacy.
Revised: As the ﬁrst step, we give the deﬁnition of pure multiplicative differential privacy.

Original: “Pure” is to distinguish from multiplicative sensitivity as deﬁned in Dwork et al.
Revised: “Pure” is to distinguish from multiplicative sensitivity as deﬁned in Dwork et al.

Original: Deﬁnition 3 (Pure multiplicative sensitivity) .
Revised: Deﬁnition 3 (Pure multiplicative sensitivity) .

Original: The multiplicative sensitivity of a function M is deﬁned as the maximum ratio between outputs on neighboring inputs S and S (cid:48) .
Revised: The multiplicative sensitivity of a function M is deﬁned as the maximum ratio between outputs on neighboring inputs S and S (cid:48) .

Original: [Equation]
Revised: [Equation]

Original: We might omit the subscript and argument when they are clear from the context.
Revised: We might omit the subscript and argument when they are clear from the context.

Original: The pure multiplicative sensitivity of F p is as below.
Revised: The pure multiplicative sensitivity of F p is as below.

Original: Theorem 1 (Multiplicative sensitivity of F p ) .
Revised: Theorem 1 (Multiplicative sensitivity of F p ) .

Original: A mechanism M which calculates F p , p ∈ (0 , 1] has pure multiplicative sensitivity upper bounded by
Revised: A mechanism M which calculates F p , p ∈ (0 , 1] has pure multiplicative sensitivity upper bounded by

Original: [Equation]
Revised: [Equation]

Original: Proof for Theorem 1.
Revised: Proof for Theorem 1.

Original: Theorem 1 gives an upper bound on the multiplicative change when two input datasets with the same size m differ in one entry.
Revised: Theorem 1 gives an upper bound on the multiplicative change when two input datasets with the same size m differ in one entry.

Original: To prove, we ﬁrst consider a slightly different setting when the second dataset is generated by adding an entry to the ﬁrst dataset.
Revised: To prove, we ﬁrst consider a slightly different setting when the second dataset is generated by adding an entry to the ﬁrst dataset.

Original: Then the neighboring datasets in the original setting can be generated by adding different entries to the same dataset.
Revised: Then the neighboring datasets in the original setting can be generated by adding different entries to the same dataset.

Original: Thus, taking the division of the upper and lower bound of the sensitivity in the incremental setting will give an upper bound for sensitivity in the original setting.
Revised: Thus, taking the division of the upper and lower bound of the sensitivity in the incremental setting will give an upper bound for sensitivity in the original setting.

Original: Concretely, let u = { u 1 , · · · , u m } where u i > 0 , (cid:80) m i =1 u i = s , ∆ ≥ 0 .
Revised: Concretely, let u = { u 1 , · · · , u m } where u i > 0 , (cid:80) m i =1 u i = s , ∆ ≥ 0 .

Original: We would like to ﬁnd both upper and lower bounds for the below expression.
Revised: We would like to ﬁnd both upper and lower bounds for the below expression.

Original: [Equation]
Revised: [Equation]

Original: To bound expression (1), we ﬁrst observe the following two inequalities (2) and (3).
Revised: To bound expression (1), we ﬁrst observe the following two inequalities (2) and (3).

Original: [Equation]
Revised: [Equation]

Original: Inequality (2) can be proved with simple algebra.
Revised: Inequality (2) can be proved with simple algebra.

Original: The left-hand-side of inequality (3) follows because (cid:80)
Revised: The left-hand-side of inequality (3) follows because (cid:80)

Original: m 1 u pi is concave in ( u 1 , . . .
Revised: m 1 u pi is concave in ( u 1 , . . .

Original: , u n ) in the simplex deﬁned by the conditions u
Revised: , u n ) in the simplex deﬁned by the conditions u

Original: i ≥ 

Revised: i ≥

Original: for all i , and (cid:80) m 1 u i = s
Revised: for all i , and (cid:80) m 1 u i = s

Original: and hence the minimum of (cid:80) m 1 u pi on the simplex is attained at a vertex of the simplex.
Revised: and hence the minimum of (cid:80) m 1 u pi on the simplex is attained at a vertex of the simplex.

Original: The right-hand-side of inequality (3) is an instance of the well-known generalized mean inequality (S`ykora, 2009) or Hölder inequality (Hölder, 1889).
Revised: The right-hand-side of inequality (3) is an instance of the well-known generalized mean inequality (S`ykora, 2009) or Hölder inequality (Hölder, 1889).

Original: First, let’s upper bound expression (1).
Revised: First, let’s upper bound expression (1).

Original: According to inequality (2) and (3), (cid:80)
Revised: According to inequality (2) and (3), (cid:80)

Original: [Equation]
Revised: [Equation]

Original: Similarly, to lower bound expression (1),
Revised: Similarly, to lower bound expression (1),

Original: [Equation]
Revised: [Equation]

Original: Taking the division between the supremum and the inﬁmum, we get
Revised: Taking the division between the supremum and the inﬁmum, we get

Original: [Equation]
Revised: [Equation]

Original: p
Revised: p

Original: In a typical streaming model where m is large and n (cid:29) M , ρ p (cid:47) 2 2 − 2 p ≤ 4 .
Revised: In a typical streaming model where m is large and n (cid:29) M , ρ p (cid:47) 2 2 − 2 p ≤ 4 .

Original: To get a better sense of how ρ changes with p , we plot several curves with different hyper-parameters in Figure 1.
Revised: To get a better sense of how ρ changes with p , we plot several curves with different hyper-parameters in Figure 1.

Original: Note that the pure multiplicative sensitivity only depends on n, m, M and p which are public information.
Revised: Note that the pure multiplicative sensitivity only depends on n, m, M and p which are public information.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 3. D IFFERENTIALLY P RIVATE F 1 S KETCH
Revised: 3. D IFFERENTIALLY P RIVATE F 1 S KETCH

Original: Instead of directly diving into the complete analysis, we ﬁrst motivate the analysis with the special case of p = 1 .
Revised: Instead of directly diving into the complete analysis, we ﬁrst motivate the analysis with the special case of p = 1 .

Original: In this case, the symmetric 1 st -stable distribution is the well-known Cauchy distribution:
Revised: In this case, the symmetric 1 st -stable distribution is the well-known Cauchy distribution:

Original: D 1 ,ζ ( x ) = 1 π · ζ ζ 2 + x 2 , and thus the analyses are signiﬁcantly simpliﬁed.
Revised: D 1 ,ζ ( x ) = 1 π · ζ ζ 2 + x 2 , and thus the analyses are signiﬁcantly simpliﬁed.

Original: The main purpose of this section is to pave the way for the proof of general F p sketch.
Revised: The main purpose of this section is to pave the way for the proof of general F p sketch.

Original: Theorem 2 ( (cid:15) -DP for F 1 sketch) .
Revised: Theorem 2 ( (cid:15) -DP for F 1 sketch) .

Original: Let ρ 1 represent the multiplicative sensitivity of the ﬁrst frequency moments.
Revised: Let ρ 1 represent the multiplicative sensitivity of the ﬁrst frequency moments.

Original: When the size of the sketch r = 1 , F 1 is ln ρ 1 -differentially private.
Revised: When the size of the sketch r = 1 , F 1 is ln ρ 1 -differentially private.

Original: Proof for Theorem 2.
Revised: Proof for Theorem 2.

Original: D 1 ,F 1 ( x ) D 1 ,ρ 1 F 1 ( x ) = ρ 21 F 21 + x 2 ρ 1 ( F 21 + x 2 ) is a decreasing function  when x ∈ (0 , ∞ ) because its derivative 2( ρ 1 − ρ 31 ) F 21 x ρ 21 ( F 21 + x 2 ) 2 is non-positive.
Revised: D 1 ,F 1 ( x ) D 1 ,ρ 1 F 1 ( x ) = ρ 21 F 21 + x 2 ρ 1 ( F 21 + x 2 ) is a decreasing function of x when x ∈ (0 , ∞ ) because its derivative 2( ρ 1 − ρ 31 ) F 21 x ρ 21 ( F 21 + x 2 ) 2 is non-positive.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: Then, for any data stream S and arbitrary measurable subset s ,
Revised: Then, for any data stream S and arbitrary measurable subset s ,

Original: [Equation]
Revised: [Equation]

Original: 3. D IFFERENTIALLY P RIVATE F p S KETCH , p ∈ (0 , 1]
Revised: 3. D IFFERENTIALLY P RIVATE F p S KETCH , p ∈ (0 , 1]

Original: The example of F 1 being (cid:15) -DP indicates the possibility that F p might have similar property when p ∈ (0 , 1] .
Revised: The example of F 1 being (cid:15) -DP indicates the possibility that F p might have similar property when p ∈ (0 , 1] .

Original: To validate, we plot the curves for different values of p s as shown in Figure 2.
Revised: To validate, we plot the curves for different values of p s as shown in Figure 2.

Original: From the ﬁgure we can tell that when p ∈ (0 , 1] , the ratio D p, 1 ( x ) D p, 2 ( x ) seems to be well-bounded and preserve (cid:15) -DP.
Revised: From the ﬁgure we can tell that when p ∈ (0 , 1] , the ratio D p, 1 ( x ) D p, 2 ( x ) seems to be well-bounded and preserve (cid:15) -DP.

Original: We now prove the conjecture as formalized in Theorem 3.
Revised: We now prove the conjecture as formalized in Theorem 3.

Original: Theorem 3 ( (cid:15) -DP for Algorithm 1) .
Revised: Theorem 3 ( (cid:15) -DP for Algorithm 1) .

Original: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.
Revised: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.

Original: When r = 1 and p ∈ (0 , 1] , F p sketch (Algorithm 1) is 1 p ln ρ p -differentially private.
Revised: When r = 1 and p ∈ (0 , 1] , F p sketch (Algorithm 1) is 1 p ln ρ p -differentially private.

Original: Proof for Theorem 3.
Revised: Proof for Theorem 3.

Original: To prove Theorem 3, we prove the following inequality.
Revised: To prove Theorem 3, we prove the following inequality.

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst prove the right-hand-side of the inequality.
Revised: We ﬁrst prove the right-hand-side of the inequality.

Original: Observe that ζ
Revised: Observe that ζ

Original: [Equation]
Revised: [Equation]

Original: = 12 π (cid:82) R cos( xt ) exp( − ζ | t | p ) dt
Revised: = 12 π (cid:82) R cos( xt ) exp( − ζ | t | p ) dt

Original: = D p,ζ ( x ) where (cid:63) substitutes t with ζ 1 p t using integration by substitution.
Revised: = D p,ζ ( x ) where (cid:63) substitutes t with ζ 1 p t using integration by substitution.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: as D p, 1 is increasing on ( −∞ , 0] and decreasing on [0 , ∞ ) , and ρ p ≥ 1 .
Revised: as D p, 1 is increasing on ( −∞ , 0] and decreasing on [0 , ∞ ) , and ρ p ≥ 1 .

Original: To prove the left-hand-side of the inequality, we reorganize it into the format of a Fourier transform.
Revised: To prove the left-hand-side of the inequality, we reorganize it into the format of a Fourier transform.

Original: [Equation]
Revised: [Equation]

Original: It sufﬁces to show that
Revised: It sufﬁces to show that

Original: [Equation]
Revised: [Equation]

Original: is decreasing.
Revised: is decreasing.

Original: Taking the ﬁrst derivative of h , we have
Revised: Taking the ﬁrst derivative of h , we have

Original: [Equation]
Revised: [Equation]

Original: ) dt , where g ( t ) = exp( − ρF p t p )( ρF p t p + 1)
Revised: , where g ( t ) = exp( − ρF p t p [Equation] ρF p t p + 1)

Original: According to Pólya criterion (Gneiting, 2001), it sufﬁces to show that g is positive deﬁnite.
Revised: According to Pólya criterion (Gneiting, 2001), it sufﬁces to show that g is positive deﬁnite.

Original: We ﬁrst observe that the function 0 ≤ u (cid:55)→ (1 + u 1 / 2 ) e − u 1 / 2 is the Laplace transform (Schiff, 1999) of the positive function 0 < t (cid:55)→ e − 1 / (4 t ) 4 √ π t 5 / 

Revised: We ﬁrst observe that the function 0 ≤ u (cid:55)→ (1 + u 1 / 2 ) e − u 1 / 2 is the Laplace transform (Schiff, 1999) of the positive function 0 < t (cid:55)→ e − 1 / (4 t ) 4 √ π t 5 /

Original: (the proof is deferred to the end) and hence a mixture of exponential functions 0 ≤ u (cid:55)→ e − cu with c > 0 .
Revised: (the proof is deferred to the end) and hence a mixture of exponential functions 0 ≤ u (cid:55)→ e − cu with c > 0 .

Original: Thus with variable substitution, the function s (cid:55)→ (1 + | s | p )
Revised: Thus with variable substitution, the function s (cid:55)→ (1 + | s | p )

Original: e −| s | p is a mixture of functions s (cid:55)→ e − c | s | 2 p with c > 0 , which are positive deﬁnite for any p ∈ (0 , 1] as they are characteristic functions of stable distributions.
Revised: e −| s | p is a mixture of functions s (cid:55)→ e − c | s | 2 p with c > 0 , which are positive deﬁnite for any p ∈ (0 , 1] as they are characteristic functions of stable distributions.

Original: The last step is to prove the function F ( u ) =
Revised: The last step is to prove the function F ( u ) =

Original: (1 + u 1 / 2 ) e − u 1 / 2 , u ≥ 0 is the Laplace transform of f ( t ) =
Revised: (1 + u 1 / 2 ) e − u 1 / 2 , u ≥ 0 is the Laplace transform of f ( t ) =

Original: e − 1 / (4 t ) 4 √ π t 5 / 2 , t > 0 : F ( u ) =
Revised: e − 1 / (4 t ) 4 √ π t 5 / 2 , t > 0 : F ( u ) =

Original: (cid:82) ∞ 

Revised: (cid:82) ∞

Original: f ( t ) e − ut dt, u ≥ 0 .
Revised: f ( t ) e − ut dt, u ≥ 0 .

Original: Let R ( u ) =
Revised: Let R ( u ) =

Original: F ( u ) and L ( u ) =
Revised: F ( u ) and L ( u ) =

Original: (cid:82) ∞ 

Revised: (cid:82) ∞

Original: f ( t ) e − ut dt .
Revised: f ( t ) e − ut dt .

Original: We observe that  L ( ∞− ) =
Revised: We observe that lim x →∞ L ( x ) =

Original:  R ( ∞− ) =
Revised: lim x →∞ R ( x ) =

Revised: lim x →∞ L (cid:48)

Original: L (cid:48) ( ∞− ) =  R (cid:48) ( ∞− ) {+= 
+} so it is enough to show that L (cid:48)(cid:48) ( u ) =
Revised: ( x ) = lim x →∞ R (cid:48) ( x ) {+= 
+} so it is enough to show that L (cid:48)(cid:48) ( u ) =

Original: R (cid:48)(cid:48) ( u ) .
Revised: R (cid:48)(cid:48) ( u ) .

Original: After a simple rescaling, it is enough to show
Revised: After a simple rescaling, it is enough to show

Revised: J

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where a > 0 as both sides do not contain linear terms.
Revised: where a > 0 as both sides do not contain linear terms.

Original: Using substitutions t
Revised: Using substitutions t

Original: = u 2 and then u
Revised: = u 2 and then u

Original: = 1 / ( x √ a ) , we get
Revised: = 1 / ( x √ a ) , we get

Original: [Equation]
Revised: [Equation]

Original: where K ( a ) :
Revised: where K ( a ) :

Original: = (cid:82) ∞ 0 exp (cid:110) − ax 2 − 1 x 2 (cid:111) dxx 2 .
Revised: = (cid:82) ∞ 0 exp (cid:110) − ax 2 − 1 x 2 (cid:111) dxx 2 .

Original: Note that K (cid:48) ( a ) =
Revised: Note that K (cid:48) ( a ) =

Original: − J ( a ) and K ( a ) =
Revised: − J ( a ) and K ( a ) =

Original: J ( a ) √ a .
Revised: J ( a ) √ a .

Original: So, we get the differential equation J (cid:48)
Revised: So, we get the differential equation J (cid:48)

Original: ( a ) = − (cid:16) 1 √ a + 12 a (cid:17)
Revised: ( a ) = − (cid:16) 1 √ a + 12 a (cid:17)

Original: J ( a ) , whose general solution is given by
Revised: J ( a ) , whose general solution is given by

Original: J ( a ) =
Revised: J ( a ) =

Revised: [Equation]

Original: c √ a e − 2 √ a for a constant c .
Revised: √ a e − 2 √ a for a constant c .

Original: To determine c , note that K ( a ) =
Revised: To determine c , note that K ( a ) =

Original: J ( a
Revised: J ( a

Original: [Equation]
Revised: [Equation]

Original: √ a = [Equation] ∞
Revised: √ a = (cid:82)

Original: [Equation]
Revised: [Equation]

Original: (cid:110)

Original: [Equation] 1 u [Equation] −
Revised: 3. P RIVACY A MPLIFICATION BY S UB - SAMPLING

Original: [Equation]
Revised: [Figure]

Original: (cid:111) [Equation] and c = K (0+ [Equation]
Revised: budget (cid:15) vs. p .

Original:  = (cid:82) [Equation] 3. P RIVACY A MPLIFICATION BY S UB - SAMPLING   p (cid:15)
Revised: n = 2 15 , m = 2 20 , M = 2 4 .

Original: The last step of Algorithm 1 estimates ζ given samples from the stable distributions.
Revised: The last step of Algorithm 1 estimates ζ given samples from the stable distributions.

Original: There are many candidate estimators such as the geometric estimator and the harmonic estimator (Li, 2008; 2009).
Revised: There are many candidate estimators such as the geometric estimator and the harmonic estimator (Li, 2008; 2009).

Original: These estimators typically, as suggested in Li (2008), require at least r ≥ 50 samples to give an accurate estimation of ζ .
Revised: These estimators typically, as suggested in Li (2008), require at least r ≥ 50 samples to give an accurate estimation of ζ .

Original: However, the privacy parameter (cid:15) grows linearly with r with trivial composition (Dwork et al., 2006), which might result in too weak privacy protection.
Revised: However, the privacy parameter (cid:15) grows linearly with r with trivial composition (Dwork et al., 2006), which might result in too weak privacy protection.

Original: To address, we follow the standard approach, amplifying privacy using sub-sampling.
Revised: To address, we follow the standard approach, amplifying privacy using sub-sampling.

Original: Different from Algorithm 1, each input has probability q to be inserted into each dimension of a , as presented in Algorithm 2.
Revised: Different from Algorithm 1, each input has probability q to be inserted into each dimension of a , as presented in Algorithm 2.

Original: If we take q = 1 r , then the privacy parameters in Theorem 3 hold as is.
Revised: If we take q = 1 r , then the privacy parameters in Theorem 3 hold as is.

Original: The proof is a simple application of the composition theorems (Dwork et al., 2006) and privacy ampliﬁcation (Balle et al., 2018).
Revised: The proof is a simple application of the composition theorems (Dwork et al., 2006) and privacy ampliﬁcation (Balle et al., 2018).

Original: Theorem 4 ( (cid:15) -DP for Algorithm 2) .
Revised: Theorem 4 ( (cid:15) -DP for Algorithm 2) .

Original: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.
Revised: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.

Original: When p ∈ (0 , 1] , F p sketch with sub-sampling rate q is qrp ln ρ p -differentially private.
Revised: When p ∈ (0 , 1] , F p sketch with sub-sampling rate q is qrp ln ρ p -differentially private.

Original: Algorithm 2: F p sketch with sub-sampling.
Revised: Algorithm 2: F p sketch with sub-sampling.

Original: The only change appears in line 3-4 and 7, corresponding to line 3 and 5 in Algorithm 1.
Revised: The only change appears in line 3-4 and 7, corresponding to line 3 and 5 in Algorithm 1.

Original: Bernoulli ( q ) refers to Bernoulli distribution with success probability q .
Revised: Bernoulli ( q ) refers to Bernoulli distribution with success probability q .

Original: Input : Data stream: S = { ( k 1 , v 1 ) , · · · , ( k n , v n ) } 1 Construction
Revised: 1 Construction

Original: Initializea = { 0 } r , P ∼ D r × m p, 1 ; for i ∈ [ n ] dob ∼ Bernoulli ( q ) ;
Revised: Initialize do

Original: Let e k
Revised: Let e k

Original: i be the one-hot encoder of k i , a = a + P × bv
Revised: i be the one-hot encoder of k i , a = a + P × bv

Original: i e k
Revised: i e k

Original: i 6 Query return
Revised: i 6 Query p ;

Original: 3. U TILITY OF A LGORITHM
Revised: U TILITY OF A LGORITHM

Original: We depict the accuracy of a F p estimator with a pair of parameters ( γ, η ) .
Revised: We depict the accuracy of a F p estimator with a pair of parameters ( γ, η ) .

Original: Deﬁnition 4 ( ( γ, η ) -Accuracy) .
Revised: Deﬁnition 4 ( ( γ, η ) -Accuracy) .

Original: A randomized algorithm M is said to be ( γ, η ) -accurate 
Revised: A randomized algorithm M is said to be ( γ, η ) -accurate if

Original: if (1 − γ ) F p ( S ) ≤ M ( S ) ≤ (1 + γ ) F p ( S ) w.p. 1 − η
Revised: [Equation]

Original: Algorithm 2 satisﬁes the following utility guarantee.
Revised: Algorithm 2 satisﬁes the following utility guarantee.

Original: The space complexity is only worse than the optimal non-private algorithm (Kane et al., 2011) by a logarithmic factor.
Revised: The space complexity is only worse than the optimal non-private algorithm (Kane et al., 2011) by a logarithmic factor.

Original: The accuracy bound is also a worst-case bound and the performance in practice is typically much better (Section 4).
Revised: The accuracy bound is also a worst-case bound and the performance in practice is typically much better (Section 4).

Original: Theorem 5 (Utility of Algorithm 2) .
Revised: Theorem 5 (Utility of Algorithm 2) .

Original: ∀ p ∈ (0 , 1] and ∀ γ, η ∈ (0 , 1) , Algorithm 2 is ( γ + (cid:113) q p − q 2 p λ , η + λ ) -accurate if r = O (cid:0) γ − 2 log( 1 η ) (cid:1) .
Revised: ∀ p ∈ (0 , 1] and ∀ γ, η ∈ (0 , 1) , Algorithm 2 is ( γ + (cid:113) q p − q 2 p λ , η + λ ) -accurate if r = O (cid:0) γ − 2 log( 1 η ) (cid:1) .

Original: In this case, Algorithm 2 uses O (cid:0) γ − 2 log( mM/ ( γη )) log( 1 η ) (cid:1) bits.
Revised: In this case, Algorithm 2 uses O (cid:0) γ − 2 log( mM/ ( γη )) log( 1 η ) (cid:1) bits.

Original: Proof.
Revised: Proof.

Original: Let SA q ( · ) represent the sub-sampling process and F rp represent a F p sketch with length r .
Revised: Let SA q ( · ) represent the sub-sampling process and F rp represent a F p sketch with length r .

Original: Then Algorithm 2 can be represented as F rp ◦ SA q where ◦ represents composition of mechanisms.
Revised: Then Algorithm 2 can be represented as F rp ◦ SA q where ◦ represents composition of mechanisms.

Original: First, we need the accuracy of F p sketch.
Revised: First, we need the accuracy of F p sketch.

Original: According to Theorem 4 of Indyk (2006), if we ﬁx the sub-sampled items,
Revised: According to Theorem 4 of Indyk (2006), if we ﬁx the sub-sampled items,

Original: [Equation]
Revised: [Equation]

Original: Second, we need the accuracy of the sub-sampling process.
Revised: Second, we need the accuracy of the sub-sampling process.

Original: The expectation and variance of the sub-sampling process is as follow.
Revised: The expectation and variance of the sub-sampling process is as follow.

Original: [Equation]
Revised: [Equation]

Original: According to Chebyshev’s inequality,
Revised: According to Chebyshev’s inequality,

Original: [Equation]
Revised: [Equation]

Original: p − q 2 p λ
Revised: p − q 2 p λ

Original: F p ( S )] ≥ 1 − λ (6)
Revised: F p ( S )] ≥ 1 − λ (6)

Original: Combining (4), (6) and (6) we get Theorem 5.
Revised: Combining (4), (6) and (6) we get Theorem 5.

Original:  E VALUATION 4. E VALUATION S
Revised:  E VALUATION 4. E VALUATION S

Original: [Figure]
Revised: [Figure]

Original: As we would like to empirically understand F p sketch’s trade-off between space, error and privacy, we evaluate F p with p ∈ { 0 . 25 , 0 . 5 , 0 .
Revised: As we would like to empirically understand F p sketch’s trade-off between space, error and privacy, we evaluate F p with p ∈ { 0 . 25 , 0 . 5 , 0 .

Original: 75 , 1 } using synthetic streams of different sizes and distributions.
Revised: 75 , 1 } using synthetic streams of different sizes and distributions.

Original: We also evaluate F p with p ∈ { 0 . 05 , 0 . 1 , · · · , 0 .
Revised: We also evaluate F p with p ∈ { 0 . 05 , 0 . 1 , · · · , 0 .

Original: 95 , 1 } on real-world data.
Revised: 95 , 1 } on real-world data.

Original: All the experiments were run on a Ubuntu18.04 LTS server with 32 AMD Opteron(TM)
Revised: All the experiments were run on a Ubuntu18.04 LTS server with 32 AMD Opteron(TM)

Original: Processor 6212 with 512GB RAM.
Revised: Processor 6212 with 512GB RAM.

Original: Synthetic Data.
Revised: Synthetic Data.

Original: We ﬁrst evaluate F p sketches using synthetic data.
Revised: We ﬁrst evaluate F p sketches using synthetic data.

Original: We synthesize two kinds of data: the key domain is either uniformly or binomially distributed.
Revised: We synthesize two kinds of data: the key domain is either uniformly or binomially distributed.

Original: The value domain is {1} by default.
Revised: The value domain is {1} by default.

Original: The size of the key domain is 1000 .
Revised: The size of the key domain is 1000 .

Original: Real-world Data.
Revised: Real-world Data.

Original: We also evaluate F p sketches using real-world application usage data (Ye et al., 2019) collected by TalkingData SDK.
Revised: We also evaluate F p sketches using real-world application usage data (Ye et al., 2019) collected by TalkingData SDK.

Original: There are more than 30 million events in this dataset, each representing one access to the TalkingData SDK.
Revised: There are more than 30 million events in this dataset, each representing one access to the TalkingData SDK.

Original: We view the event type as the key and the value is set to 1 by default.
Revised: We view the event type as the key and the value is set to 1 by default.

Original: 4. E VALUATION R ESULTS
Revised: 4. E VALUATION R ESULTS

Original: In this section, we present the evaluation results.
Revised: In this section, we present the evaluation results.

Original: To avoid the inﬂuence of outliers, we report the median and interquartile of 100 runs for each data point except for the real-data evaluation.
Revised: To avoid the inﬂuence of outliers, we report the median and interquartile of 100 runs for each data point except for the real-data evaluation.

Original: For all the evaluation, the sketch size r is 50 as suggested in Li (2008).
Revised: For all the evaluation, the sketch size r is 50 as suggested in Li (2008).

Original: The sub-sampling rate in all the experiments is 0 .
Revised: The sub-sampling rate in all the experiments is 0 .

Original: Synthetic Data.
Revised: Synthetic Data.

Original: The evaluation results on synthetic data are presented in Figure 4.
Revised: The evaluation results on synthetic data are presented in Figure 4.

Original: For uniformly distributed data, we observe that as the stream size increases, the multiplicative error decreases.
Revised: For uniformly distributed data, we observe that as the stream size increases, the multiplicative error decreases.

Original: We conjecture the reason to be the effect of sub-sampling.
Revised: We conjecture the reason to be the effect of sub-sampling.

Original: Concretely, each bin in the value domain has to get enough samples to approximate the behavior of the true distribution.
Revised: Concretely, each bin in the value domain has to get enough samples to approximate the behavior of the true distribution.

Original: On the other hand, when the data is binomially distributed, the multiplicative error is relatively stable with small ﬂuctuation.
Revised: On the other hand, when the data is binomially distributed, the multiplicative error is relatively stable with small ﬂuctuation.

Original: We conjecture the reason is that as binomial distribution is more concentrated, the sample complexity is smaller than uniform distribution.
Revised: We conjecture the reason is that as binomial distribution is more concentrated, the sample complexity is smaller than uniform distribution.

Original: Besides, for uniformly distributed data, p s close to 0 have relatively large errors while the errors when p is close to 1 are small.
Revised: Besides, for uniformly distributed data, p s close to 0 have relatively large errors while the errors when p is close to 1 are small.

Original: The reason is that the further p is from 1 , the larger the inﬂuence of sub-sampling.
Revised: The reason is that the further p is from 1 , the larger the inﬂuence of sub-sampling.

Original: [Figure]
Revised: [Figure]

Original: E rr o r γ
Revised: E rr o r γ

Original: [Figure]
Revised: [Figure]

Original: x ) D p, 2 p ( x ) with different values of p ∈ (1 , 2) on R + .
Revised: x ) D p, 2 p ( x ) with different values of p ∈ (1 , 2) on R + .

Original: The negative half is symmetric.
Revised: The negative half is symmetric.

Original: The x-axis is log-scale to highlight the complex monotone trends.
Revised: The x-axis is log-scale to highlight the complex monotone trends.

Original: Real-world Data.
Revised: Real-world Data.

Original: The evaluation results for real-world data are presented in Figure 5.
Revised: The evaluation results for real-world data are presented in Figure 5.

Original: We sampled 100,0000 data points from the dataset and the key has a domain of size 1488095.
Revised: We sampled 100,0000 data points from the dataset and the key has a domain of size 1488095.

Original: Each data point is the median of 5 runs.
Revised: Each data point is the median of 5 runs.

Original: We observe that the further p is from 1 , the higher the multiplicative error.
Revised: We observe that the further p is from 1 , the higher the multiplicative error.

Original: This conforms with our observation in the evaluation on synthetic data.
Revised: This conforms with our observation in the evaluation on synthetic data.

Original:  C ONCLUSION & F UTURE W ORK
Revised:  C ONCLUSION & F UTURE W ORK

Original: This paper takes an important step towards narrowing the gap of space complexity between private and non-private frequency moments estimation algorithms.
Revised: This paper takes an important step towards narrowing the gap of space complexity between private and non-private frequency moments estimation algorithms.

Original: We prove that F p is differentially private as is when p ∈ (0 , 1] and thus give the ﬁrst differentially private frequency estimation protocol with polylogarithmic space complexity.
Revised: We prove that F p is differentially private as is when p ∈ (0 , 1] and thus give the ﬁrst differentially private frequency estimation protocol with polylogarithmic space complexity.

Original: At the same time, we observe several open challenges.
Revised: At the same time, we observe several open challenges.

Original: First, the proof does not easily extend to p ∈ (1 , 2) .
Revised: First, the proof does not easily extend to p ∈ (1 , 2) .

Original: Figure 6 exhibits the complexity of monocity of D p, 1 ( x ) D p, 2 p ( x ) when p ∈ (1 , 2) .
Revised: Figure 6 exhibits the complexity of monocity of D p, 1 ( x ) D p, 2 p ( x ) when p ∈ (1 , 2) .

Original: The most complex curve when p = 1 . 99 is composed of three monotonic parts in the ﬁgure.
Revised: The most complex curve when p = 1 . 99 is composed of three monotonic parts in the ﬁgure.

Original: Hence, an interesting next step is to fully understand the monotonicity pattern of the ratio curve when p ∈ (1 , 2) and get corresponding privacy parameters.
Revised: Hence, an interesting next step is to fully understand the monotonicity pattern of the ratio curve when p ∈ (1 , 2) and get corresponding privacy parameters.

Original: Second, the space complexity of Algorithm 2 is still worse than the optimal non-private algorithm by a factor of log( m ) .
Revised: Second, the space complexity of Algorithm 2 is still worse than the optimal non-private algorithm by a factor of log( m ) .

Original: It is interesting to check whether the optimal algorithm (Kane et al., 2011) also preserves differential privacy.
Revised: It is interesting to check whether the optimal algorithm (Kane et al., 2011) also preserves differential privacy.

Original: Ethics Statement.
Revised: Ethics Statement.

Original: Our work study the intrinsic differential privacy of F p sketches.
Revised: Our work study the intrinsic differential privacy of F p sketches.

Original: F p sketches should be used with careful calibration to make sure the output is accurate and provides reasonable privacy guarantee.
Revised: F p sketches should be used with careful calibration to make sure the output is accurate and provides reasonable privacy guarantee.

Original: All the datasets and packages used are open-sourced.
Revised: All the datasets and packages used are open-sourced.