Original: D IFFERENTIALLY P RIVATE F RACTIONAL F REQUENCY M OMENTS E STIMATION WITH P OLYLOGARITHMIC S PACE
Revised: D IFFERENTIALLY P RIVATE F RACTIONAL F REQUENCY M OMENTS E STIMATION WITH P OLYLOGARITHMIC S PACE

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We prove that F p sketch, a well-celebrated streaming algorithm for frequency moments estimation, is differentially private as is when p ∈ (0 , 1] .
Revised: We prove that F p sketch, a well-celebrated streaming algorithm for frequency moments estimation, is differentially private as is when p ∈ (0 , 1] .

Original: F p sketch uses only polylogarithmic space, exponentially better than existing DP baselines and only worse than the optimal non-private baseline by a logarithmic factor.
Revised: F p sketch uses only polylogarithmic space, exponentially better than existing DP baselines and only worse than the optimal non-private baseline by a logarithmic factor.

Original: The evaluation shows that F p sketch can achieve reasonable accuracy with differential privacy guarantee.
Revised: The evaluation shows that F p sketch can achieve reasonable accuracy with differential privacy guarantee.

Original: The evaluation code is included in the supplementary material.
Revised: The evaluation code is included in the supplementary material.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Counting is one of the most fundamental operations in almost every area of computer science.
Revised: Counting is one of the most fundamental operations in almost every area of computer science.

Original: It typically refers to estimating the cardinality (the 0 th frequency moment) of a given set.
Revised: It typically refers to estimating the cardinality (the 0 th frequency moment) of a given set.

Original: However, counting can actually refer to the process of estimating a broader class of statistics, namely p th frequency moment, denoted F p .
Revised: However, counting can actually refer to the process of estimating a broader class of statistics, namely p th frequency moment, denoted F p .

Original: Frequency moments estimation is at the core of various important statistical problems.
Revised: Frequency moments estimation is at the core of various important statistical problems.

Original: F 1 is used for data mining (Cormode et al., 2005) and hypothesis tests (Indyk & McGregor, 2008).
Revised: F 1 is used for data mining (Cormode et al., 2005) and hypothesis tests (Indyk & McGregor, 2008).

Original: F 2 has applications in calculating Gini index (Lorenz, 1905; Gini, 1912) and surprise index (Good, 1989), training random forests (Breiman, 2001), numerical linear algebra (Clarkson & Woodruff, 2009; Sarlos, 2006) and network anomaly detection (Krishnamurthy et al., 2003; Thorup & Zhang, 2004).
Revised: F 2 has applications in calculating Gini index (Lorenz, 1905; Gini, 1912) and surprise index (Good, 1989), training random forests (Breiman, 2001), numerical linear algebra (Clarkson & Woodruff, 2009; Sarlos, 2006) and network anomaly detection (Krishnamurthy et al., 2003; Thorup & Zhang, 2004).

Original: Fractional frequency moments are used in Shannon entropy estimation (Harvey et al., 2008; Zhao et al., 2007) and image decomposition (Geiger et al., 1999).
Revised: Fractional frequency moments are used in Shannon entropy estimation (Harvey et al., 2008; Zhao et al., 2007) and image decomposition (Geiger et al., 1999).

Original: Non-private frequency moments estimation is systematically studied in the data streaming model (Alon et al., 1999;
Revised: Non-private frequency moments estimation is systematically studied in the data streaming model (Alon et al., 1999;

Original: Charikar et al., 2002; Thorup & Zhang, 2004; Feigenbaum et al., 2002;
Revised: Charikar et al., 2002; Thorup & Zhang, 2004; Feigenbaum et al., 2002;

Original: Indyk, 2006; Li, 2008; Kane et al., 2010; Nelson & Woodruff, 2009; 2010; Kane et al., 2011).
Revised: Indyk, 2006; Li, 2008; Kane et al., 2010; Nelson & Woodruff, 2009; 2010; Kane et al., 2011).

Original: This model assumes extremely limited storage such as network routers.
Revised: This model assumes extremely limited storage such as network routers.

Original: The optimal non-private algorithm (Kane et al., 2011) uses only polylogarithmic space to maintain frequency moments.
Revised: The optimal non-private algorithm (Kane et al., 2011) uses only polylogarithmic space to maintain frequency moments.

Original: In the present work, we inherit the low space complexity requirement for the versatility of the algorithm.
Revised: In the present work, we inherit the low space complexity requirement for the versatility of the algorithm.

Original: The data being counted sometimes contains sensitive information.
Revised: The data being counted sometimes contains sensitive information.

Original: For example, to calculate Gini index, the data should contain pairs of ID and income.
Revised: For example, to calculate Gini index, the data should contain pairs of ID and income.

Original: Frequency moments of such data, if published, might leak sensitive information.
Revised: Frequency moments of such data, if published, might leak sensitive information.

Original: To mitigate, the gold standard of differential privacy (DP) should be applied.
Revised: To mitigate, the gold standard of differential privacy (DP) should be applied.

Original: Special cases of DP frequency moments estimation such as p = 0 , 1 , 2 are well-studied in a wide spectrum of works (Choi et al., 2020; Smith et al., 2020;
Revised: Special cases of DP frequency moments estimation such as p = 0 , 1 , 2 are well-studied in a wide spectrum of works (Choi et al., 2020; Smith et al., 2020;

Original: Blocki et al., 2012; Sheffet, 2017;
Revised: Blocki et al., 2012; Sheffet, 2017;

Original: Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021; Mir et al., 2011).
Revised: Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021; Mir et al., 2011).

Original: In the present work, we make the ﬁrst customized effort towards DP estimation of fractional frequency moments, i.e. p ∈ (0 , 1] with low space complexity.
Revised: In the present work, we make the ﬁrst customized effort towards DP estimation of fractional frequency moments, i.e. p ∈ (0 , 1] with low space complexity.

Original: We show that a well-known streaming algorithm, namely F p sketch (Indyk, 2006), preserves differential privacy as is.
Revised: We show that a well-known streaming algorithm, namely F p sketch (Indyk, 2006), preserves differential privacy as is.

Original: With its small space complexity, F p sketch elegantly solves the trilemma between efﬁciency, accuracy, and privacy.
Revised: With its small space complexity, F p sketch elegantly solves the trilemma between efﬁciency, accuracy, and privacy.

Original: Problem Formulation.
Revised: Problem Formulation.

Original: We use bold lowercase letters to denote vectors ( e.g. a , b , c ) and bold uppercase letters to denote matrices ( e.g. A , B , C ).
Revised: We use bold lowercase letters to denote vectors ( e.g. a , b , c ) and bold uppercase letters to denote matrices ( e.g. A , B , C ).

Original: { 1 , · · · , n } is denoted by [ n ] .
Revised: { 1 , · · · , n } is denoted by [ n ] .

Original: Let S
Revised: Let S

Original: =
Revised: =

Original: ( k 1 , v 1 ) , · · · , ( k n , v n ) } ( n ≥ 1) be a stream of key-value pairs where k i ∈
Revised: ( k 1 , v 1 ) , · · · , ( k n , v n ) } ( n ≥ 1) be a stream of key-value pairs where k i ∈

Original: [ m ] ( m ≥ 2) , v i ∈
Revised: [ m ] ( m ≥ 2) , v i ∈

Original: [ M ] ( M ≥ 1) 1 .
Revised: [ M ] ( M ≥ 1) 1 .

Original: We would like to design a randomized mechanism M that estimates the p th frequency moment:
Revised: We would like to design a randomized mechanism M that estimates the p th frequency moment:

Original: [Equation]
Revised: [Equation]

Original: for p ∈ (0 , 1] where I is an indicator function returning 1 if k = k
Revised: for p ∈ (0 , 1] where I is an indicator function returning 1 if k = k

Original: i
Revised: i

Original: and 0 otherwise.
Revised: and 0 otherwise.

Original: To provide rigorous privacy guarantee, M should preserve differential privacy as deﬁned below.
Revised: To provide rigorous privacy guarantee, M should preserve differential privacy as deﬁned below.

Original: In our setting, neighboring data streams differ in one key-value pair.
Revised: In our setting, neighboring data streams differ in one key-value pair.

Original: Deﬁnition 1 ( ( (cid:15), δ ) -Differential Privacy) .
Revised: Deﬁnition 1 ( ( (cid:15), δ ) -Differential Privacy) .

Original: A randomized algorithm M is said to preserve ( (cid:15), δ ) -DP
Revised: A randomized algorithm M is said to preserve ( (cid:15), δ ) -DP

Original: if for two neighboring datasets S , S (cid:48) and any measurable subset of the output space s ,
Revised: if for two neighboring datasets S , S (cid:48) and any measurable subset of the output space s ,

Original: [Equation]
Revised: [Equation]

Original: When δ = 0 , we omit it and denote the privacy guarantee as (cid:15) -DP.
Revised: When δ = 0 , we omit it and denote the privacy guarantee as (cid:15) -DP.

Original: Oftentimes, n, m is large ( e.g. IP streams on routers)
Revised: Oftentimes, n, m is large ( e.g. IP streams on routers)

Original: so M should take polylogarithmic space in terms of n, m .
Revised: so M should take polylogarithmic space in terms of n, m .

Original: Proof Intuition.
Revised: Proof Intuition.

Original: We summarize the intuition behind the proof that F p sketch is differentially private when p ∈ (0 , 1] .
Revised: We summarize the intuition behind the proof that F p sketch is differentially private when p ∈ (0 , 1] .

Original: Recall that when proving DP for traditional mechanisms such as the Gaussian mechanism, the core is to upper-bound the ratio P ( x ) Q ( x ) where P ( x ) and Q ( x ) are the probability density functions of outputs when the inputs are neighboring datasets.
Revised: Recall that when proving DP for traditional mechanisms such as the Gaussian mechanism, the core is to upper-bound the ratio P ( x ) Q ( x ) where P ( x ) and Q ( x ) are the probability density functions of outputs when the inputs are neighboring datasets.

Original: In the proof of Gaussian mechanism, P ( x ) and Q ( x ) can be viewed as a horizontal translation of each other and the distance between their mean values is the sensitivity of the output.
Revised: In the proof of Gaussian mechanism, P ( x ) and Q ( x ) can be viewed as a horizontal translation of each other and the distance between their mean values is the sensitivity of the output.

Original: For F p , however, neighboring inputs do not translate the output distribution but instead change its scale.
Revised: For F p , however, neighboring inputs do not translate the output distribution but instead change its scale.

Original: For example, when p = 2 , P ( x ) and Q ( x ) are Gaussian distributions with the same mean and different variance.
Revised: For example, when p = 2 , P ( x ) and Q ( x ) are Gaussian distributions with the same mean and different variance.

Original: Inspired by the analogy to Gaussian mechanism, we need to address the below two questions to prove differential privacy for F p sketches.
Revised: Inspired by the analogy to Gaussian mechanism, we need to address the below two questions to prove differential privacy for F p sketches.

Original: Q1.
Revised: Q1.

Original: How to bound the difference between the scales of P ( x ) and Q ( x ) ?
Revised: How to bound the difference between the scales of P ( x ) and Q ( x ) ?

Original: Q2.
Revised: Q2.

Original: How to bound the ratio between the density functions of P ( x ) and Q ( x ) ?
Revised: How to bound the ratio between the density functions of P ( x ) and Q ( x ) ?

Original: To answer Q1, we propose a new sensitivity deﬁnition called pure multiplicative sensitivity .
Revised: To answer Q1, we propose a new sensitivity deﬁnition called pure multiplicative sensitivity .

Original: Pure multiplicative sensitivity depicts the maximal multiplicative change in the output when the inputs are neighboring datasets.
Revised: Pure multiplicative sensitivity depicts the maximal multiplicative change in the output when the inputs are neighboring datasets.

Original: We analyze frequency moments estimation and ﬁnd that its pure multiplicative sensitivity is approximately max { 2 2 p − 2 , 2 2 − 2 p } when p ∈ (0 , 1] and n (cid:29) M .
Revised: We analyze frequency moments estimation and ﬁnd that its pure multiplicative sensitivity is approximately max { 2 2 p − 2 , 2 2 − 2 p } when p ∈ (0 , 1] and n (cid:29) M .

Original: To answer Q2, we ﬁrst revisit the special case of p = 1 .
Revised: To answer Q2, we ﬁrst analyze the special case of p = 1 .

Original: As shown by Mir et al.

Original: (2011), when p = 1 , P ( x ) Q ( x ) is rigorously upper-bounded and thus F 1 sketch preserves (cid:15)
Revised: When p = 1 , P ( x ) Q ( x ) is rigorously upperbounded and thus F 1 sketch preserves (cid:15)

Original: -DP.
Revised: -DP.

Original: By analogy, we conjecture that F p , p ∈ (0 , 1] also satisﬁes similar properties, which is doubly conﬁrmed by the numerically simulated plots in Figure 2.
Revised: By analogy, we conjecture that F p , p ∈ (0 , 1] also satisﬁes similar properties, which is doubly conﬁrmed by the numerically simulated plots in Figure 2.

Original: The conjecture is formally proved in Theorem 3.
Revised: The conjecture is formally proved in Theorem 3.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Frequency moments estimation is thoroughly studied in the data streaming model.
Revised: Frequency moments estimation is thoroughly studied in the data streaming model.

Original: Alon et al.
Revised: Alon et al.

Original: (1999) proposed the ﬁrst space-efﬁcient algorithm for estimating p th frequency moments when p is integer.
Revised: (1999) proposed the ﬁrst space-efﬁcient algorithm for estimating p th frequency moments when p is integer.

Original: Indyk (2006) extended the use case from integer moments to fractional moments using stable distributions.
Revised: Indyk (2006) extended the use case from integer moments to fractional moments using stable distributions.

Original: A line of following works improve Indyk’s algorithm in various aspects such as space complexity (Kane et al., 2010; Nelson & Woodruff, 2009), time complexity (Nelson & Woodruff, 2010; Kane et al., 2011) or accuracy (Li, 2008; 2009).
Revised: A line of following works improve Indyk’s algorithm in various aspects such as space complexity (Kane et al., 2010; Nelson & Woodruff, 2009), time complexity (Nelson & Woodruff, 2010; Kane et al., 2011) or accuracy (Li, 2008; 2009).

Original: Several special cases in private frequency moments estimation such as p = 0 , 1 , 2 were also well studied.
Revised: Several special cases in private frequency moments estimation such as p = 0 , 1 , 2 were also well studied.

Original: Choi et al.
Revised: Choi et al.

Original: (2020) and Smith et al.
Revised: (2020) and Smith et al.

Original: (2020) studied differentially private F 0 estimation.
Revised: (2020) studied differentially private F 0 estimation.

Original: They separately proved that the Flajolet-Martin sketch is differentially private as is.
Revised: They separately proved that the Flajolet-Martin sketch is differentially private as is.

Original: Several independent works (Blocki et al., 2012; Sheffet, 2017; Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021) studied the differential privacy guarantee in the special case p = 2 under the name of Johnson-Lindenstrauss projection.
Revised: Several independent works (Blocki et al., 2012; Sheffet, 2017; Upadhyay, 2014; Choi et al., 2020; Bu et al., 2021) studied the differential privacy guarantee in the special case p = 2 under the name of Johnson-Lindenstrauss projection.

Original: On the other hand, there is barely any prior work focusing on differentially private fractional frequency moments estimation.
Revised: On the other hand, there is barely any prior work focusing on differentially private fractional frequency moments estimation.

Original: Differentially private distribution estimation algorithms (Acs et al., 2012; Xu et al., 2013; Bassily & Smith, 2015; Suresh, 2019; Wang et al., 2019) can be used to provide a differentially private estimation of fractional frequency moments.
Revised: Differentially private distribution estimation algorithms (Acs et al., 2012; Xu et al., 2013; Bassily & Smith, 2015; Suresh, 2019; Wang et al., 2019) can be used to provide a differentially private estimation of fractional frequency moments.

Original: However, they are overkill as their outputs contain much more information than the queried fractional frequency moment.
Revised: However, they are overkill as their outputs contain much more information than the queried fractional frequency moment.

Original: They only provide sub-optimal privacy-utility trade-off and are exponentially worse in terms of space complexity.
Revised: They only provide sub-optimal privacy-utility trade-off and are exponentially worse in terms of space complexity.

Original: Datar et al.
Revised: Datar et al.

Original: (2004) considered a similar (but not the same) mathematical problem to the present work when designing a locality-sensitive hashing scheme.
Revised: (2004) considered a similar (but not the same) mathematical problem to the present work when designing a locality-sensitive hashing scheme.

Original: However, their analysis focuses on the simple cases when p = 1 and p = 2 and totally depends on numerical analysis for p ∈ (0 , 1) .
Revised: However, their analysis focuses on the simple cases when p = 1 and p = 2 and totally depends on numerical analysis for p ∈ (0 , 1) .

Original:  D IFFERENTIALLY P RIVATE F REQUENCY M OMENTS E STIMATION
Revised:  D IFFERENTIALLY P RIVATE F REQUENCY M OMENTS E STIMATION

Original: In this section, we ﬁrst revisit F p sketch and then prove the differential privacy guarantee for F p sketch step by step.
Revised: In this section, we ﬁrst revisit F p sketch and then prove the differential privacy guarantee for F p sketch step by step.

Original: Different from most differential privacy analyses based on additive sensitivity, our proof depends on a variant of the multiplicative sensitivity (Dwork et al., 2015) called pure multiplicative sensitivity .
Revised: Different from most differential privacy analyses based on additive sensitivity, our proof depends on a variant of the multiplicative sensitivity (Dwork et al., 2015) called pure multiplicative sensitivity .

Original: We give the ﬁrst analysis of pure multiplicative sensitivity for p -th frequency moments.
Revised: We give the ﬁrst analysis of pure multiplicative sensitivity for p -th frequency moments.

Original: Then we motivate the differential privacy proof using a special case when p = 1 .
Revised: Then we motivate the differential privacy proof using a special case when p = 1 .

Original: Finally we proceed to the general proof that F p sketch preserves differential privacy.
Revised: Finally we proceed to the general proof that F p sketch preserves differential privacy.

Original: The main challenge stems from the fact that the density functions of p -stable distributions have no close-form expressions when p ∈ (0 , 1) .
Revised: The main challenge stems from the fact that the density functions of p -stable distributions have no close-form expressions when p ∈ (0 , 1) .

Original: 3. R EVISITING F p S KETCH
Revised: 3. R EVISITING F p S KETCH

Original: For completeness, we revisit the well-celebrated F p sketch by Indyk (2006) (also known as stable projection or compressed counting).
Revised: For completeness, we revisit the well-celebrated F p sketch by Indyk (2006) (also known as stable projection or compressed counting).

Original: We ﬁrst introduce p -stable distribution, the basic building block in F p sketch.
Revised: We ﬁrst introduce p -stable distribution, the basic building block in F p sketch.

Original: Then we review how to construct and query F p sketch using stable distributions.
Revised: Then we review how to construct and query F p sketch using stable distributions.

Original: Deﬁnition 2 ( p -stable distribution) .
Revised: Deﬁnition 2 ( p -stable distribution) .

Original: A random variable X follows a β -skewed p -stable distribution if its characteristic function is
Revised: A random variable X follows a β -skewed p -stable distribution if its characteristic function is

Original: [Equation]
Revised: [Equation]

Original: where − 1 ≤ β ≤ 1 is the skewness parameter, ζ > 0 is the scale parameter to the p
Revised: where − 1 ≤ β ≤ 1 is the skewness parameter, ζ > 0 is the scale parameter to the p

Original: th power.
Revised: th power.

Original: In this paper, we focus on stable distributions with β = 0 , namely symmetric stable distributions.
Revised: In this paper, we focus on stable distributions with β = 0 , namely symmetric stable distributions.

Original: We denote a symmetric p -stable distribution by D p,ζ , and slightly abuse the notation to denote the density function as D p,ζ ( x ) .
Revised: We denote a symmetric p -stable distribution by D p,ζ , and slightly abuse the notation to denote the density function as D p,ζ ( x ) .

Original: Note that the density function is the inverse Fourier transform of the characteristic function.
Revised: Note that the density function is the inverse Fourier transform of the characteristic function.

Original: [Equation]
Revised: [Equation]

Original: If two independent random variables X 1 , X 2 ∼ D p, 1 , then C 1 X 1 + C 

Revised: If two independent random variables X 1 , X 2 ∼ D p, 1 , then C 1 X 1 + C

Original: X
Revised: X

Original: 2 ∼ D p,C p 1 + C p 2 .
Revised: 2 ∼ D p,C p 1 + C p 2 .

Original: We refer to this property as p -stability .
Revised: We refer to this property as p -stability .

Original: F p sketch leverages the p -stability of these distributions to keep track of the frequency moments.
Revised: F p sketch leverages the p -stability of these distributions to keep track of the frequency moments.

Original: The pseudo-code for vanilla F p sketch is presented in Algorithm 1.
Revised: The pseudo-code for vanilla F p sketch is presented in Algorithm 1.

Original: To construct, a sketch of size r is initialized to all zeros and a projection matrix P is sampled from D r × m p,(line 2).
Revised: To construct, a sketch of size r is initialized to all zeros and a projection matrix P is sampled from D r × m p,(line 2).

Original: For each incoming key-value pair ( k i , v i ) , we multiply the one-hot encoding of k i scaled by v i with the projection matrix
Revised: For each incoming key-value pair ( k i , v i ) , we multiply the one-hot encoding of k i scaled by v i with the projection matrix

Original: P and add it to the sketch (line 4).
Revised: P and add it to the sketch (line 4).

Original: [Equation]
Revised: [Equation]

Original: To query the sketch, we estimate ζ from a using various estimators such as median, inter-quantile, geometric mean or harmonic mean as suggested by Indyk (2006), Li (2008) and Li (2009).
Revised: To query the sketch, we estimate ζ from a using various estimators such as median, inter-quantile, geometric mean or harmonic mean as suggested by Indyk (2006), Li (2008) and Li (2009).

Original: Algorithm1:
Revised: Algorithm

Original: 3. P URE MULTIPLICATIVE SENSITIVITY OF FREQUENCY MOMENTS ESTIMATION
Revised: 3. P URE MULTIPLICATIVE SENSITIVITY OF FREQUENCY MOMENTS ESTIMATION

Original: As we will see in the following two subsections, the differential privacy proof for F p sketch depends on the pure multiplicative sensitivity of p -th frequency moments.
Revised: As we will see in the following two subsections, the differential privacy proof for F p sketch depends on the pure multiplicative sensitivity of p -th frequency moments.

Original: As the ﬁrst step, we give the deﬁnition of pure multiplicative differential privacy.
Revised: As the ﬁrst step, we give the deﬁnition of pure multiplicative differential privacy.

Original: “Pure” is to distinguish from multiplicative sensitivity as deﬁned in Dwork et al.
Revised: “Pure” is to distinguish from multiplicative sensitivity as deﬁned in Dwork et al.

Original: Deﬁnition 3 (Pure multiplicative sensitivity) .
Revised: Deﬁnition 3 (Pure multiplicative sensitivity) .

Original: The multiplicative sensitivity of a function M is deﬁned as the maximum ratio between outputs on neighboring inputs S and S (cid:48) .
Revised: The multiplicative sensitivity of a function M is deﬁned as the maximum ratio between outputs on neighboring inputs S and S (cid:48) .

Original: [Equation]
Revised: [Equation]

Original: We might omit the subscript and argument when they are clear from the context.
Revised: We might omit the subscript and argument when they are clear from the context.

Original: The pure multiplicative sensitivity of F p is as below.
Revised: The pure multiplicative sensitivity of F p is as below.

Original: Theorem 1 (Multiplicative sensitivity of F p ) .
Revised: Theorem 1 (Multiplicative sensitivity of F p ) .

Original: A mechanism M which calculates F p , p ∈ (0 , 1] has pure multiplicative sensitivity upper bounded by
Revised: A mechanism M which calculates F p , p ∈ (0 , 1] has pure multiplicative sensitivity upper bounded by

Original: [Equation]
Revised: [Equation]

Original: Proof for Theorem 1.
Revised: Proof for Theorem 1.

Original: Theorem 1 gives an upper bound on the multiplicative change when two input datasets with the same size m differ in one entry.
Revised: Theorem 1 gives an upper bound on the multiplicative change when two input datasets with the same size m differ in one entry.

Original: To prove, we ﬁrst consider a slightly different setting when the second dataset is generated by adding an entry to the ﬁrst dataset.
Revised: To prove, we ﬁrst consider a slightly different setting when the second dataset is generated by adding an entry to the ﬁrst dataset.

Original: Then the neighboring datasets in the original setting can be generated by adding different entries to the same dataset.
Revised: Then the neighboring datasets in the original setting can be generated by adding different entries to the same dataset.

Original: Thus, taking the division of the upper and lower bound of the sensitivity in the incremental setting will give an upper bound for sensitivity in the original setting.
Revised: Thus, taking the division of the upper and lower bound of the sensitivity in the incremental setting will give an upper bound for sensitivity in the original setting.

Original: Concretely, let u = { u 1 , · · · , u m } where u i > 0 , (cid:80) m i =1 u i = s , ∆ ≥ 0 .
Revised: Concretely, let u = { u 1 , · · · , u m } where u i > 0 , (cid:80) m i =1 u i = s , ∆ ≥ 0 .

Original: We would like to ﬁnd both upper and lower bounds for the below expression.
Revised: We would like to ﬁnd both upper and lower bounds for the below expression.

Original: [Equation]
Revised: [Equation]

Original: To bound expression (1), we ﬁrst observe the following two inequalities (2) and (3).
Revised: To bound expression (1), we ﬁrst observe the following two inequalities (2) and (3).

Original: [Equation]
Revised: [Equation]

Original: Inequality (2) can be proved with simple algebra.
Revised: Inequality (2) can be proved with simple algebra.

Original: The left-hand-side of inequality (3) follows because (cid:80)
Revised: The left-hand-side of inequality (3) follows because (cid:80)

Original: m 1 u pi is concave in ( u 1 , . . .
Revised: m 1 u pi is concave in ( u 1 , . . .

Original: , u n ) in the simplex deﬁned by the conditions u
Revised: , u n ) in the simplex deﬁned by the conditions u

Original: i ≥ 

Revised: i ≥

Original: for all i , and (cid:80) m 1 u i = s
Revised: for all i , and (cid:80) m 1 u i = s

Original: and hence the minimum of (cid:80) m 1 u pi on the simplex is attained at a vertex of the simplex.
Revised: and hence the minimum of (cid:80) m 1 u pi on the simplex is attained at a vertex of the simplex.

Original: The right-hand-side of inequality (3) is an instance of the well-known generalized mean inequality (S`ykora, 2009) or Hölder inequality (Hölder, 1889).
Revised: The right-hand-side of inequality (3) is an instance of the well-known generalized mean inequality (S`ykora, 2009) or Hölder inequality (Hölder, 1889).

Original: First, let’s upper bound expression (1).
Revised: First, let’s upper bound expression (1).

Original: According to inequality (2) and (3),(cid:80)
Revised: According to inequality (2) and (3),

Original: [Equation]
Revised: [Equation]

Original: Similarly, to lower bound expression (1),
Revised: Similarly, to lower bound expression (1),

Original: [Equation]
Revised: [Equation]

Original: Taking the division between the supremum and the inﬁmum, we get
Revised: Taking the division between the supremum and the inﬁmum, we get

Original: [Equation]
Revised: [Equation]

Original: p In a typical streaming model where m is large and n (cid:29) M , ρ p (cid:47) 2 2 − 2 p ≤ 4 .
Revised: In a typical streaming model where m is large and n (cid:29) M , ρ p (cid:47) 2 2 − 2 p ≤ 4 .

Original: To get a better sense of how ρ changes with p , we plot several curves with different hyper-parameters in Figure 1.
Revised: To get a better sense of how ρ changes with p , we plot several curves with different hyper-parameters in Figure 1.

Original: Note that the pure multiplicative sensitivity only depends on n, m, M and p which are public information.
Revised: Note that the pure multiplicative sensitivity only depends on n, m, M and p which are public information.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 3. D IFFERENTIALLY P RIVATE F 1 S KETCH
Revised: 3. D IFFERENTIALLY P RIVATE F 1 S KETCH

Original: Instead of directly diving into the complete analysis, we ﬁrst motivate the analysis with the special case of p = 1 .
Revised: Instead of directly diving into the complete analysis, we ﬁrst motivate the analysis with the special case of p = 1 .

Original: In this case, the symmetric 1 st -stable distribution is the well-known Cauchy distribution:
Revised: In this case, the symmetric 1 st -stable distribution is the well-known Cauchy distribution:

Original: D 1 ,ζ ( x ) = 1 π · ζ ζ 2 + x 2 , and thus the analyses are signiﬁcantly simpliﬁed.
Revised: D 1 ,ζ ( x ) = 1 π · ζ ζ 2 + x 2 , and thus the analyses are signiﬁcantly simpliﬁed.

Original: The main purpose of this section is to pave the way for the proof of general F p sketch.
Revised: The main purpose of this section is to pave the way for the proof of general F p sketch.

Original: Theorem 2 ( (cid:15) -DP for F 1 sketch) .
Revised: Theorem 2 ( (cid:15) -DP for F 1 sketch) .

Original: Let ρ 1 represent the multiplicative sensitivity of the ﬁrst frequency moments.
Revised: Let ρ 1 represent the multiplicative sensitivity of the ﬁrst frequency moments.

Original: When the size of the sketch r = 1 , F 1 is ln ρ 1 -differentially private.
Revised: When the size of the sketch r = 1 , F 1 is ln ρ 1 -differentially private.

Original: Proof for Theorem 2.
Revised: Proof for Theorem 2.

Original: D 1 ,F 1 ( x ) D 1 ,ρ 1 F 1 ( x ) = ρ 21 F 21 + x 2 ρ 1 ( F 21 + x 2 ) is a decreasing function of x when x ∈ (0 , ∞ ) because its derivative 2( ρ 1 − ρ 31 ) F 21 x ρ 21 ( F 21 + x 2 ) 2 is non-positive.
Revised: D 1 ,F 1 ( x ) D 1 ,ρ 1 F 1 ( x ) = ρ 21 F 21 + x 2 ρ 1 ( F 21 + x 2 ) is a decreasing function of x when x ∈ (0 , ∞ ) because its derivative 2( ρ 1 − ρ 31 ) F 21 x ρ 21 ( F 21 + x 2 ) 2 is non-positive.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: Then, for any data stream S and arbitrary measurable subset s ,
Revised: Then, for any data stream S and arbitrary measurable subset s ,

Original: [Equation]
Revised: [Equation]

Original: 3. D IFFERENTIALLY P RIVATE F p S KETCH , p ∈ (0 , 1]
Revised: 3. D IFFERENTIALLY P RIVATE F p S KETCH , p ∈ (0 , 1]

Original: The example of F 1 being (cid:15) -DP indicates the possibility that F p might have similar property when p ∈ (0 , 1] .
Revised: The example of F 1 being (cid:15) -DP indicates the possibility that F p might have similar property when p ∈ (0 , 1] .

Original: To validate, we plot the curves for different values of p s as shown in Figure 2.
Revised: To validate, we plot the curves for different values of p s as shown in Figure 2.

Original: From the ﬁgure we can tell that when p ∈ (0 , 1] , the ratio D p, 1 ( x ) D p, 2 ( x ) seems to be well-bounded and preserve (cid:15) -DP.
Revised: From the ﬁgure we can tell that when p ∈ (0 , 1] , the ratio D p, 1 ( x ) D p, 2 ( x ) seems to be well-bounded and preserve (cid:15) -DP.

Original: We now prove the conjecture as formalized in Theorem 3.
Revised: We now prove the conjecture as formalized in Theorem 3.

Original: Theorem 3 ( (cid:15) -DP for Algorithm 1) .
Revised: Theorem 3 ( (cid:15) -DP for Algorithm 1) .

Original: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.
Revised: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.

Original: When r = 1 and p ∈ (0 , 1] , F p sketch (Algorithm 1) is 1 p ln ρ p -differentially private.
Revised: When r = 1 and p ∈ (0 , 1] , F p sketch (Algorithm 1) is 1 p ln ρ p -differentially private.

Original: Proof for Theorem 3.
Revised: Proof for Theorem 3.

Original: To prove Theorem 3, we prove the following inequality.
Revised: To prove Theorem 3, we prove the following inequality.

Original: [Equation]
Revised: [Equation]

Original: We ﬁrst prove the right-hand-side of the inequality.
Revised: We ﬁrst prove the right-hand-side of the inequality.

Original: Observe thatζ
Revised: Observe that

Original: [Equation]
Revised: [Equation]

Original: = 12 π (cid:82) R cos( xt ) exp( − ζ | t | p ) dt
Revised: = 12 π (cid:82) R cos( xt ) exp( − ζ | t | p ) dt

Original: = D p,ζ ( x ) where (cid:63) substitutes t with ζ 1 p t using integration by substitution.
Revised: = D p,ζ ( x ) where (cid:63) substitutes t with ζ 1 p t using integration by substitution.

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: as D p, 1 is increasing on ( −∞ , 0] and decreasing on [0 , ∞ ) , and ρ p ≥ 1 .
Revised: as D p, 1 is increasing on ( −∞ , 0] and decreasing on [0 , ∞ ) , and ρ p ≥ 1 .

Original: To prove the left-hand-side of the inequality, we reorganize it into the format of a Fourier transform.
Revised: To prove the left-hand-side of the inequality, we reorganize it into the format of a Fourier transform.

Original: [Equation]
Revised: [Equation]

Original: It sufﬁces to show that
Revised: It sufﬁces to show that

Original: [Equation]
Revised: [Equation]

Original: is decreasing.
Revised: is decreasing.

Original: Taking the ﬁrst derivative of h , we have
Revised: Taking the ﬁrst derivative of h , we have

Original: [Equation]
Revised: [Equation]

Original: , where g ( t ) = exp( − ρF p t p
Revised: , where g ( t ) = exp [Equation] − ρF p t p

Original: [Equation]
Revised: [Equation]

Original: ρF p t p+ 1)
Revised: ρF p [Equation] p

Original:  According to Pólya criterion (Gneiting, 2001), it sufﬁces to show that g is positive deﬁnite.
Revised: [Equation] According to Pólya criterion (Gneiting, 2001), it sufﬁces to show that g is positive deﬁnite.

Original: We ﬁrst observe that the function 0 ≤ u (cid:55)→ (1 + u 1 / 2 ) e − u 1 / 2 is the Laplace transform (Schiff, 1999) of the positive function 0 < t (cid:55)→ e − 1 / (4 t ) 4 √ π t 5 / 

Revised: We ﬁrst observe that the function 0 ≤ u (cid:55)→ (1 + u 1 / 2 ) e − u 1 / 2 is the Laplace transform (Schiff, 1999) of the positive function 0 < t (cid:55)→ e − 1 / (4 t ) 4 √ π t 5 /

Original: (the proof is deferred to the end) and hence a mixture of exponential functions 0 ≤ u (cid:55)→ e − cu with c > 0 .
Revised: (the proof is deferred to the end) and hence a mixture of exponential functions 0 ≤ u (cid:55)→ e − cu with c > 0 .

Original: Thus with variable substitution, the function s (cid:55)→ (1 + | s | p )
Revised: Thus with variable substitution, the function s (cid:55)→ (1 + | s | p )

Original: e −| s | p is a mixture of functions s (cid:55)→ e − c | s | 2 p with c > 0 , which are positive deﬁnite for any p ∈ (0 , 1] as they are characteristic functions of stable distributions.
Revised: e −| s | p is a mixture of functions s (cid:55)→ e − c | s | 2 p with c > 0 , which are positive deﬁnite for any p ∈ (0 , 1] as they are characteristic functions of stable distributions.

Original: The last step is to prove the function F ( u ) =
Revised: The last step is to prove the function F ( u ) =

Original: (1 + u 1 / 2 ) e − u 1 / 2 , u ≥ 0 is the Laplace transform of f ( t ) =
Revised: (1 + u 1 / 2 ) e − u 1 / 2 , u ≥ 0 is the Laplace transform of f ( t ) =

Original: e − 1 / (4 t ) 4 √ π t 5 / 2 , t > 0 : F ( u ) =
Revised: e − 1 / (4 t ) 4 √ π t 5 / 2 , t > 0 : F ( u ) =

Original: (cid:82) ∞ 

Revised: (cid:82) ∞

Original: f ( t ) e − ut dt, u ≥ 0 .
Revised: f ( t ) e − ut dt, u ≥ 0 .

Original: Let R ( u ) =
Revised: Let R ( u ) =

Original: F ( u ) and
Revised: F ( u ) and

Original: L ( u ) =
Revised: L ( u ) =

Original: (cid:82) ∞ 

Revised: (cid:82) ∞

Original: f ( t ) e − ut dt .
Revised: f ( t ) e − ut dt .

Original: We observe that lim x →∞ L ( x ) =
Revised: We observe that lim x →∞ L ( x ) =

Original: lim x →∞ R ( x ) =
Revised: lim x →∞ R ( x ) =

Original: lim x →∞ L (cid:48)
Revised: lim x →∞ L (cid:48)

Original: ( x ) =
Revised: ( x ) =

Original: lim x →∞ R (cid:48)
Revised: lim x →∞ R (cid:48)

Original: ( x ) = 

Revised: ( x ) =

Original: so it is enough to show that L (cid:48)(cid:48) ( u ) =
Revised: so it is enough to show that L (cid:48)(cid:48) ( u ) =

Original: R (cid:48)(cid:48) ( u ) .
Revised: R (cid:48)(cid:48) ( u ) .

Original: After a simple rescaling, it is enough to show
Revised: After a simple rescaling, it is enough to show

Original: J [Equation]
Revised: J ( a ) :

Original: [Equation]
Revised: [Equation]

Original: where a > 0 as both sides do not contain linear terms.
Revised: where a > 0 as both sides do not contain linear terms.

Original: Using substitutions t
Revised: Using substitutions t

Original: = u 2 and then u
Revised: = u 2 and then u

Original: = 1 / ( x √ a ) , we get
Revised: = 1 / ( x √ a ) , we get

Original: [Equation]
Revised: [Equation]

Original: where K ( a ) :
Revised: where K ( a ) :

Original: = (cid:82) ∞ 0 exp (cid:110) − ax 2 − 1 x 2 (cid:111) dxx 2 .
Revised: = (cid:82) ∞ 0 exp (cid:110) − ax 2 − 1 x 2 (cid:111) dxx 2 .

Original: Note that K (cid:48) ( a ) =
Revised: Note that K (cid:48) ( a ) =

Original: − J ( a ) and K ( a ) =
Revised: − J ( a ) and K ( a ) =

Original: J ( a ) √ a .
Revised: J ( a ) √ a .

Original: So, we get the differential equation J (cid:48)
Revised: So, we get the differential equation J (cid:48)

Original: ( a ) = − (cid:16) 1 √ a + 12 a (cid:17)
Revised: ( a ) = − (cid:16) 1 √ a + 12 a (cid:17)

Original: J ( a ) , whose general solution is given by
Revised: J ( a ) , whose general solution is given by

Original: J ( a ) =
Revised: J ( a ) =

Original: [Equation]
Revised: [Equation]

Original: √ a e − 2 √ a for a constant c .
Revised: √ a e − 2 √ a for a constant c .

Original: To determine c , note that K ( a ) =
Revised: To determine c , note that K ( a ) =

Original: J ( a [Equation] √ a = (cid:82)
Revised: J ( a ) √ a = (cid:82)

Original: [Equation]
Revised: [Equation]

Original: 3. P RIVACY A MPLIFICATION BY S UB - SAMPLING
Revised: 3. P RIVACY A MPLIFICATION BY S UB - SAMPLING

Original: [Figure]
Revised: [Figure]

Original: budget (cid:15) vs. p .
Revised: (cid:15) vs. p .

Original: n = 2 15 , m = 2 20 , M = 2 4 .
Revised: n = 2 15 , m = 2 20 , M = 2 4 .

Original: The last step of Algorithm 1 estimates ζ given samples from the stable distributions.
Revised: The last step of Algorithm 1 estimates ζ given samples from the stable distributions.

Original: There are many candidate estimators such as the geometric estimator and the harmonic estimator (Li, 2008; 2009).
Revised: There are many candidate estimators such as the geometric estimator and the harmonic estimator (Li, 2008; 2009).

Original: These estimators typically, as suggested in Li (2008), require at least r ≥ 50 samples to give an accurate estimation of ζ .
Revised: These estimators typically, as suggested in Li (2008), require at least r ≥ 50 samples to give an accurate estimation of ζ .

Original: However, the privacy parameter (cid:15) grows linearly with r with trivial composition (Dwork et al., 2006), which might result in too weak privacy protection.
Revised: However, the privacy parameter (cid:15) grows linearly with r with trivial composition (Dwork et al., 2006), which might result in too weak privacy protection.

Original: To address, we follow the standard approach, amplifying privacy using sub-sampling.
Revised: To address, we follow the standard approach, amplifying privacy using sub-sampling.

Original: Different from Algorithm 1, each input has probability q to be inserted into each dimension of a , as presented in Algorithm 2.
Revised: Different from Algorithm 1, each input has probability q to be inserted into each dimension of a , as presented in Algorithm 2.

Original: If we take q = 1 r , then the privacy parameters in Theorem 3 hold as is.
Revised: If we take q = 1 r , then the privacy parameters in Theorem 3 hold as is.

Original: The proof is a simple application of the composition theorems (Dwork et al., 2006) and privacy ampliﬁcation (Balle et al., 2018).
Revised: The proof is a simple application of the composition theorems (Dwork et al., 2006) and privacy ampliﬁcation (Balle et al., 2018).

Original: Theorem 4 ( (cid:15) -DP for Algorithm 2) .
Revised: Theorem 4 ( (cid:15) -DP for Algorithm 2) .

Original: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.
Revised: Let ρ p represent the multiplicative sensitivity of the p -th frequency moments.

Original: When p ∈ (0 , 1] , F p sketch with sub-sampling rate q is qrp ln ρ p -differentially private.
Revised: When p ∈ (0 , 1] , F p sketch with sub-sampling rate q is qrp ln ρ p -differentially private.

Original: Algorithm 2: F p sketch with sub-sampling.
Revised: Algorithm 2: F p sketch with sub-sampling.

Original: The only change appears in line 3-4 and 7, corresponding to line 3 and 5 in Algorithm 1.
Revised: The only change appears in line 3-4 and 7, corresponding to line 3 and 5 in Algorithm 1.

Original: Bernoulli ( q ) refers to Bernoulli distribution with success probability q . 
Revised: Bernoulli ( q ) refers to Bernoulli distribution with success probability q . Input

Original: 1 Construction
Revised: 1 Construction

Original: Initializedo
Revised: Initialize

Revised: ; for do b

Original: Let e k
Revised: Let e k

Original: i be the one-hot encoder of k i , a = a + P × bv
Revised: i be the one-hot encoder of k i , a = a + P × bv

Original: i e k
Revised: i e k

Original: i 6 Query  p ; 
Revised: i 6 Query return a ) /q p ; 3.

Original: UTILITY OF A LGORITHM
Revised: U

Revised: A LGORITHM

Original: We depict the accuracy of a F p estimator with a pair of parameters ( γ, η ) .
Revised: We depict the accuracy of a F p estimator with a pair of parameters ( γ, η ) .

Original: Deﬁnition 4 ( ( γ, η ) -Accuracy) .
Revised: Deﬁnition 4 ( ( γ, η ) -Accuracy) .

Original: A randomized algorithm M is said to be ( γ, η ) -accurate if
Revised: A randomized algorithm M is said to be ( γ, η ) -accurate if

Original: [Equation]
Revised: [Equation]

Original: Algorithm 2 satisﬁes the following utility guarantee.
Revised: Algorithm 2 satisﬁes the following utility guarantee.

Original: The space complexity is only worse than the optimal non-private algorithm (Kane et al., 2011) by a logarithmic factor.
Revised: The space complexity is only worse than the optimal non-private algorithm (Kane et al., 2011) by a logarithmic factor.

Original: The accuracy bound is also a worst-case bound and the performance in practice is typically much better (Section 4).
Revised: The accuracy bound is also a worst-case bound and the performance in practice is typically much better (Section 4).

Original: Theorem 5 (Utility of Algorithm 2) .
Revised: Theorem 5 (Utility of Algorithm 2) .

Original: ∀ p ∈ (0 , 1] and ∀ γ, η ∈ (0 , 1) , Algorithm 2 is ( γ + (cid:113) q p − q 2 p λ , η + λ ) -accurate if r = O (cid:0) γ − 2 log( 1 η ) (cid:1) .
Revised: ∀ p ∈ (0 , 1] and ∀ γ, η ∈ (0 , 1) , Algorithm 2 is ( γ + (cid:113) q p − q 2 p λ , η + λ ) -accurate if r = O (cid:0) γ − 2 log( 1 η ) (cid:1) .

Original: In this case, Algorithm 2 uses O (cid:0) γ − 2 log( mM/ ( γη )) log( 1 η ) (cid:1) bits.
Revised: In this case, Algorithm 2 uses O (cid:0) γ − 2 log( mM/ ( γη )) log( 1 η ) (cid:1) bits.

Original: Proof.
Revised: Proof.

Original: Let SA q ( · ) represent the sub-sampling process and F rp represent a F p sketch with length r .
Revised: Let SA q ( · ) represent the sub-sampling process and F rp represent a F p sketch with length r .

Original: Then Algorithm 2 can be represented as F rp ◦ SA q where ◦ represents composition of mechanisms.
Revised: Then Algorithm 2 can be represented as F rp ◦ SA q where ◦ represents composition of mechanisms.

Original: First, we need the accuracy of F p sketch.
Revised: First, we need the accuracy of F p sketch.

Original: According to Theorem 4 of Indyk (2006), if we ﬁx the sub-sampled items,
Revised: According to Theorem 4 of Indyk (2006), if we ﬁx the sub-sampled items,

Original: [Equation]
Revised: [Equation]

Original: Second, we need the accuracy of the sub-sampling process.
Revised: Second, we need the accuracy of the sub-sampling process.

Original: The expectation and variance of the sub-sampling process is as follow.
Revised: The expectation and variance of the sub-sampling process is as follow.

Original: [Equation]
Revised: [Equation]

Original: According to Chebyshev’s inequality,
Revised: According to Chebyshev’s inequality,

Original: [Equation]
Revised: [Equation]

Original: p − q 2 p λ
Revised: − q 2 p λ

Original: F p ( S )] ≥ 1 − λ (6)
Revised: F p ( S )] ≥ 1 − λ (6)

Original: Combining (4), (6) and (6) we get Theorem 5.
Revised: Combining (4), (6) and (6) we get Theorem 5.

Original:  E VALUATION 4. E VALUATION S
Revised:  E VALUATION 4. E VALUATION S

Original: [Figure]
Revised: [Figure]

Original: As we would like to empirically understand F p sketch’s trade-off between space, error and privacy, we evaluate F p with p ∈ { 0 . 25 , 0 . 5 , 0 .
Revised: As we would like to empirically understand F p sketch’s trade-off between space, error and privacy, we evaluate F p with p ∈ { 0 . 25 , 0 . 5 , 0 .

Original: 75 , 1 } using synthetic streams of different sizes and distributions.
Revised: 75 , 1 } using synthetic streams of different sizes and distributions.

Original: We also evaluate F p with p ∈ { 0 . 05 , 0 . 1 , · · · , 0 .
Revised: We also evaluate F p with p ∈ { 0 . 05 , 0 . 1 , · · · , 0 .

Original: 95 , 1 } on real-world data.
Revised: 95 , 1 } on real-world data.

Original: All the experiments were run on a Ubuntu18.04 LTS server with 32 AMD Opteron(TM)
Revised: All the experiments were run on a Ubuntu18.04 LTS server with 32 AMD Opteron(TM)

Original: Processor 6212 with 512GB RAM.
Revised: Processor 6212 with 512GB RAM.

Original: Synthetic Data.
Revised: Synthetic Data.

Original: We ﬁrst evaluate F p sketches using synthetic data.
Revised: We ﬁrst evaluate F p sketches using synthetic data.

Original: We synthesize two kinds of data: the key domain is either uniformly or binomially distributed.
Revised: We synthesize two kinds of data: the key domain is either uniformly or binomially distributed.

Original: The value domain is {1} by default.
Revised: The value domain is {1} by default.

Original: The size of the key domain is 1000 .
Revised: The size of the key domain is 1000 .

Original: Real-world Data.
Revised: Real-world Data.

Original: We also evaluate F p sketches using real-world application usage data (Ye et al., 2019) collected by TalkingData SDK.
Revised: We also evaluate F p sketches using real-world application usage data (Ye et al., 2019) collected by TalkingData SDK.

Original: There are more than 30 million events in this dataset, each representing one access to the TalkingData SDK.
Revised: There are more than 30 million events in this dataset, each representing one access to the TalkingData SDK.

Original: We view the event type as the key and the value is set to 1 by default.
Revised: We view the event type as the key and the value is set to 1 by default.

Original: 4. E VALUATION R ESULTS
Revised: 4. E VALUATION R ESULTS

Original: In this section, we present the evaluation results.
Revised: In this section, we present the evaluation results.

Original: To avoid the inﬂuence of outliers, we report the median and interquartile of 100 runs for each data point except for the real-data evaluation.
Revised: To avoid the inﬂuence of outliers, we report the median and interquartile of 100 runs for each data point except for the real-data evaluation.

Original: For all the evaluation, the sketch size r is 50 as suggested in Li (2008).
Revised: For all the evaluation, the sketch size r is 50 as suggested in Li (2008).

Original: The sub-sampling rate in all the experiments is 0 .
Revised: The sub-sampling rate in all the experiments is 0 .

Original: Synthetic Data.
Revised: Synthetic Data.

Original: The evaluation results on synthetic data are presented in Figure 4.
Revised: The evaluation results on synthetic data are presented in Figure 4.

Original: For uniformly distributed data, we observe that as the stream size increases, the multiplicative error decreases.
Revised: For uniformly distributed data, we observe that as the stream size increases, the multiplicative error decreases.

Original: We conjecture the reason to be the effect of sub-sampling.
Revised: We conjecture the reason to be the effect of sub-sampling.

Original: Concretely, each bin in the value domain has to get enough samples to approximate the behavior of the true distribution.
Revised: Concretely, each bin in the value domain has to get enough samples to approximate the behavior of the true distribution.

Original: On the other hand, when the data is binomially distributed, the multiplicative error is relatively stable with small ﬂuctuation.
Revised: On the other hand, when the data is binomially distributed, the multiplicative error is relatively stable with small ﬂuctuation.

Original: We conjecture the reason is that as binomial distribution is more concentrated, the sample complexity is smaller than uniform distribution.
Revised: We conjecture the reason is that as binomial distribution is more concentrated, the sample complexity is smaller than uniform distribution.

Original: Besides, for uniformly distributed data, p s close to 0 have relatively large errors while the errors when p is close to 1 are small.
Revised: Besides, for uniformly distributed data, p s close to 0 have relatively large errors while the errors when p is close to 1 are small.

Original: The reason is that the further p is from 1 , the larger the inﬂuence of sub-sampling.
Revised: The reason is that the further p is from 1 , the larger the inﬂuence of sub-sampling.

Original: [Figure]
Revised: [Figure]

Original: E rr o r γ
Revised: γ

Original: [Figure]
Revised: [Figure]

Original: x ) D p, 2 p ( x ) with different values of p ∈ (1 , 2) on R + .
Revised: ) D p, 2 p ( x ) with different values of p ∈ (1 , 2) on R + .

Original: The negative half is symmetric.
Revised: The negative half is symmetric.

Original: The x-axis is log-scale to highlight the complex monotone trends.
Revised: The x-axis is log-scale to highlight the complex monotone trends.

Original: Real-world Data.
Revised: Real-world Data.

Original: The evaluation results for real-world data are presented in Figure 5.
Revised: The evaluation results for real-world data are presented in Figure 5.

Original: We sampled 100,0000 data points from the dataset and the key has a domain of size 1488095.
Revised: We sampled 100,0000 data points from the dataset and the key has a domain of size 1488095.

Original: Each data point is the median of 5 runs.
Revised: Each data point is the median of 5 runs.

Original: We observe that the further p is from 1 , the higher the multiplicative error.
Revised: We observe that the further p is from 1 , the higher the multiplicative error.

Original: This conforms with our observation in the evaluation on synthetic data.
Revised: This conforms with our observation in the evaluation on synthetic data.

Original:  C ONCLUSION & F UTURE W ORK
Revised:  C ONCLUSION & F UTURE W ORK

Original: This paper takes an important step towards narrowing the gap of space complexity between private and non-private frequency moments estimation algorithms.
Revised: This paper takes an important step towards narrowing the gap of space complexity between private and non-private frequency moments estimation algorithms.

Original: We prove that F p is differentially private as is when p ∈ (0 , 1] and thus give the ﬁrst differentially private frequency estimation protocol with polylogarithmic space complexity.
Revised: We prove that F p is differentially private as is when p ∈ (0 , 1] and thus give the ﬁrst differentially private frequency estimation protocol with polylogarithmic space complexity.

Original: At the same time, we observe several open challenges.
Revised: At the same time, we observe several open challenges.

Original: First, the proof does not easily extend to p ∈ (1 , 2) .
Revised: First, the proof does not easily extend to p ∈ (1 , 2) .

Original: Figure 6 exhibits the complexity of monocity of D p, 1 ( x ) D p, 2 p ( x ) when p ∈ (1 , 2) .
Revised: Figure 6 exhibits the complexity of monocity of D p, 1 ( x ) D p, 2 p ( x ) when p ∈ (1 , 2) .

Original: The most complex curve when p = 1 . 99 is composed of three monotonic parts in the ﬁgure.
Revised: The most complex curve when p = 1 . 99 is composed of three monotonic parts in the ﬁgure.

Original: Hence, an interesting next step is to fully understand the monotonicity pattern of the ratio curve when p ∈ (1 , 2) and get corresponding privacy parameters.
Revised: Hence, an interesting next step is to fully understand the monotonicity pattern of the ratio curve when p ∈ (1 , 2) and get corresponding privacy parameters.

Original: Second, the space complexity of Algorithm 2 is still worse than the optimal non-private algorithm by a factor of log( m ) .
Revised: Second, the space complexity of Algorithm 2 is still worse than the optimal non-private algorithm by a factor of log( m ) .

Original: It is interesting to check whether the optimal algorithm (Kane et al., 2011) also preserves differential privacy.
Revised: It is interesting to check whether the optimal algorithm (Kane et al., 2011) also preserves differential privacy.

Original: Ethics Statement.
Revised: Ethics Statement.

Original: Our work study the intrinsic differential privacy of F p sketches.
Revised: Our work study the intrinsic differential privacy of F p sketches.

Original: F p sketches should be used with careful calibration to make sure the output is accurate and provides reasonable privacy guarantee.
Revised: F p sketches should be used with careful calibration to make sure the output is accurate and provides reasonable privacy guarantee.

Original: All the datasets and packages used are open-sourced.
Revised: All the datasets and packages used are open-sourced.

Revised: A P URE M ULTIPLICATIVE S ENSITIVITY IN S TRICT T URNSTILE M ODEL

Revised: In this section, we derive the pure multiplicative sensitivity of F p in the strict turnstile model.

Revised: In the strict turnstile model, for a key-value stream S = { ( k 1 , v 1 ) , · · · , ( k n , v n ) } ( n ≥ 1) where k i ∈ [ m ] ( m ≥ 2) , v i ∈ {− M, · · · , M } ( M ≥ 1) , the sum of v s of the same key should always be non-negative:

Revised: [Equation]

Revised: Besides, for the utility of the result, we need to assume that M < n − 1 .

Revised: Theorem 6 (Multiplicative sensitivity of F p in strict turnstile model) .

Revised: A mechanism M < n −which calculates F p , p ∈ (0 , 1] in the strict turnstile model when has pure multiplicative sensitivity upper bounded by

Revised: [Equation]

Revised: Proof for Theorem 6.

Revised: An upper bound for the sensitivity of F p in the strict turnstile model can be derived by taking the division of the upper and lower bound in the incremental setting following the same logic as the proof for Theorem 1.

Revised: The upper bound is the same as in the proof of Theorem 6 so we only need to calculate the lower bound of the following expression.

Revised: First, we observe the following two inequalities.

Revised: [Equation]

Revised: Taking the division between the supremum and the inﬁmum, we get

Revised: [Equation]

Revised: As shown in Figure 7, when m is the same, the sensitivity is very close to the sensitivity in the cash register model if M (cid:28) n .

Revised: [Figure]

Revised: r e

Revised: [Figure]

Revised: [Equation]

Revised: [Figure]

Revised: [Equation]

Revised: ti

Revised: [Figure]

Revised: [Equation]

Revised: n

Revised: [Equation]