Title: Faster approximate subgraph counts with privacy
Venue: NeurIPS 2023
Accepted: yes
Keywords: differential privacy

subgraph counting

smooth sensitivity

local sensitivity
Abstract: One of the most common problems studied in the context of differential privacy for graph data is counting the number of non-induced embeddings of a subgraph in a given graph. 
These counts have very high global sensitivity. Therefore, adding noise based on powerful alternative techniques, such as smooth sensitivity and higher-order local sensitivity have been shown to give significantly better accuracy. 
However, all these alternatives to global sensitivity become computationally very expensive, and to date efficient polynomial time algorithms are known only for few selected subgraphs, such as triangles, $k$-triangles, and $k$-stars.
In this paper, we show that good approximations to these sensitivity metrics can be still used to get private algorithms.
Using this approach, we much faster algorithms for privately counting the number of triangles in real-world social networks, which can be easily parallelized.
We also give a private polynomial time algorithm for counting any constant size subgraph using less noise than the global sensitivity; we show this can be improved significantly for counting paths in special classes of graphs.

Faster approximate subgraph counts with privacy

Abstract

One of the most common problems studied in the context of differential privacy for graph data is counting the number of non-induced embeddings of a subgraph in a given graph. These counts have very high global sensitivity. Therefore, adding noise based on powerful alternative techniques, such as smooth sensitivity and higher-order local sensitivity have been shown to give significantly better accuracy. However, all these alternatives to global sensitivity become computationally very expensive, and to date efficient polynomial time algorithms are known only for few selected subgraphs, such as triangles, k -triangles, and k -stars. In this paper, we show that good approximations to these sensitivity metrics can be still used to get private algorithms. Using this approach, we much faster algorithms for privately counting the number of triangles in real-world social networks, which can be easily parallelized. We also give a private polynomial time algorithm for counting any constant size subgraph using less noise than the global sensitivity; we show this can be improved significantly for counting paths in special classes of graphs.

1 Introduction

The notion of Differential Privacy (DP) [12] has emerged as the de facto standard notion for supporting queries on private and sensitive data. DP ensures that changes in private data have limited statistical influence (measured by the privacy budget ϵ, δ ) on the output of queries, without needing any attack model, which is one of the reasons of its popularity. Networked abstractions are commonly used in a number of applications, such as public health, social networks, and finance. Such datasets are usually sensitive, and maintaining privacy is an important concern. Within the context of network/graph data, two privacy models have been considered in most prior work, namely, edge and node privacy. There has been a lot of interest in developing private algorithms for a number of network problems, such as community detection [29, 9, 8, 22], and counting small subgraphs (e.g., stars and triangles [25, 36, 23, 31]).

However, efficient private algorithms with good accuracy are not known for many graph problems, including counting small subgraphs. One of the main challenges is that graph problems often have very high global sensitivity (the maximum change in the output function due to the change in one edge/vertex (see Section 3)), in contrast to many statistical and machine learning queries. As a result, DP algorithms based on global sensitivity (which can usually be computed quite easily for many problems) do not give good accuracy bounds, in general. For instance, in the edge-privacy

model, the global sensitivity of #triangles in a graph G = ( V, E ) can be as high as n -2 , where n = | V | , making the added noise too large in many instances. A number of novel alternatives to global sensitivity have been proposed, e.g., smooth, restricted and multi-level local sensitivity, and ladder functions [36, 23, 3, 31], which add significantly lower level of noise than mechanisms based on global sensitivity.

However, all these alternative notions of sensitivity are computationally much more challenging, and none of the current graph DP algorithms for subgraph counting scale to even moderate size networks. Even the simplest problem of counting #triangles privately takes min { m max v d ( v ) , M ( n ) } time [23], where m = | E | , d ( v ) is the degree of node v ∈ V , and M ( n ) is the time for multiplying two n × n matrices, which is super-quadratic, in general. This is a sharp contrast with the significant advances in non-private graph mining where complex network properties such as clique counts can be computed using provably-efficient and practical algorithms [35, 20, 19]. For a few other subgraphs, such as k -cliques and k -triangles (see [36, 23] for definitions), private algorithms using noise lower than the global sensitivity are known, but the worst case running time can still be O ( n k ) [36]. However, for most subgraph counting problems, no private algorithms which use alternatives to global sensitivity, and have time comparable to the non-private algorithms, are known .

A main tool for subgraph counting and other graph mining tasks in massive networks are sampling based approximation algorithms, e.g., [2]; however, these haven't been used in the context of private graph algorithms. In this paper, we take the first steps in this direction. Our main contributions are:

· We show that for a query f ( G ) , we can get a private algorithm by adding noise that scales with an approximation to the smooth sensitivity to an approximation to f ( G ) (Theorem 1). This result opens the door to improving efficiency of private algorithms using approximation techniques that have been very useful in non-private graph analytics. As an illustration, we use this result to obtain a quasilinear time private algorithm for counting #triangles under edge privacy (Theorem 2) for graphs which satisfy a stronger transitivity property (see Definition 5). Our algorithm adapts the diamond sampling technique of [2] for approximate triangle counting, and can also be parallelized easily. No parallel algorithms have been developed for private subgraph counting so far.

· Extending the approach of [23], we design a private higher-order local sensitivity approach for subgraph counting, which gives privacy even when the higher-order local sensitivities are estimated approximately (Theorem 7). This yields the first private algorithm to count the number of embeddings of any subgraph with ℓ edges in time O ( n 2 ℓ ) , using less noise than global sensitivity. For the specific case of paths with ℓ edges, we develop an algorithm that gives a very close approximation to the higher-order sensitivity in time O ( h ( d, t, ℓ ) poly ( n )) , where d and t are the degeneracy and treewidth of G , respectively (defined later). Thus, for paths, the running time does not grow as O ( n ℓ ) for graphs with bounded degeneracy and treewidth. These properties have been used extensively in developing efficient algorithms for subgraph counting and other graph mining tasks, e.g., [5, 13, 6]; but, they haven't been used for private algorithms.

· We experimentally evaluate our algorithm for counting #triangles (Section 6). We show that our algorithm for estimating the smooth sensitivity has very good approximation factor, and significantly better running time than the exact algorithm. Using approximate smooth sensitivity for counting #triangles gives good accuracy even for fairly low ϵ values, suggesting that using approximate smooth sensitivity does not compromise performance. Our algorithm also has good scaling, and is the first private algorithm for counting #triangles which has been run on networks with over 2 million edges.

We note that many details, including proofs are presented in the Supplementary Material (SM).

2 Related Work

The area of DP graph algorithms is quite large and active. There has been a lot of work on DP algorithms for many basic graph problems, such as community detection, subgraph counting, finding small cuts, and releasing synthetic graphs [25, 28, 30, 33, 16, 3, 15, 21, 17]. There is some work on graph algorithms under local DP, e.g., [16], but most of the prior work has been for global DP (as defined in Section 3), and is our focus in this work. We refer to [27] for a recent survey on private graph algorithms. For brevity, we only discuss prior work on private subgraph counting that is directly related to our paper, and in the edge-DP model. We omit the discussion on private estimation of edge

density and degree distribution, which has been studied more extensively, e.g., [14], and results for node-DP [25], which is less studied.

As mentioned earlier, global sensitivity, denoted by GS ( f ) for a graph metric f ( G ) (defined in Section 3) is high for many subgraph counting problems. Adding noise based on GS ( f ) leads to low accuracy, while noise based on just LS f , the local sensitivity, need not be private [31]. Nissim et al. [31] develop the notion of smooth sensitivity S ∗ f, β ( G ) = max G ′ ( LS f ( G ′ ) · e -β · d ( G,G ′ ) ) , where d ( G,G ′ ) is the swap distance between graphs G and G ′ , and show that adding noise based on S ∗ f, β ( G ) is private. However, smooth sensitivity becomes computationally much more challenging, since its definition involves considering local sensitivity at distance t for all t , which doesn't seem like a polynomial time computation. Nissim et al. [31] develop polynomial time algorithms for computing the smooth sensitivity exactly for counting #triangles, which was improved slightly to min { md max , M ( n ) } [23], where M ( n ) is the matrix multiplication time. Polynomial time smooth sensitivity bounds were also shown for a small number of other subgraphs, but no polynomial time algorithms are known beyond that. Interestingly, no hardness bounds are known for smooth sensitivity; the existing hardness bounds, e.g., [23, 36] only give hardness for computing local sensitivity at distance t . In order to handle other subgraph counts, Karwa et al. [23] develop a different technique involving local sensitivity of local sensitivity (motivated by the propose-test-release technique [12]), and use it for private counts of k -cliques. Zhang et al. [36] develop the technique of ladder functions to handle other kinds of subgraphs, and use it to privately count # k -cliques in the graph in time O ( nT ( n )) , where T ( n ) is the time needed to count # k -cliques non-privately. A few other techniques, such as inverse sensitivity [1] and propose-test-release [12] are known. However, private algorithms for counting most subgraphs including paths and trees are not known. The recent work of [4] is related, but only considers approximation in the queries, but not in sensitivity. As a result, our methods give significantly higher efficiency. We also note that Blocki et al. [4] develop a black-box approach to make certain approximation algorithms differentially private. However, their work requires the function being computed to have 'small' global sensitivity.

The difficulty of subgraph counting with DP has motivated work in slight variations of the DP model. Rastogi et al. [34] consider the problems of releasing more general subgraph counts. However, they consider a relaxed version of edge-DP, called (edge) adversarial privacy, that uses a Bayesian attacker. Chen et al. [7] design a different approach that gives lower bounds for general subgraph counts through a linear program to form a recursive strategy. But, as mentioned in [36], their method suffers from a bias between the true query answer and the lower bound, in exchange for less noise. [32] presents a different approach based on iterative refinement that estimates counts by degree, and can be implemented in time O ((max v deg ( v )) 3 m ) .

There have been studies on subgraph counting (including triangle counting) under other privacy models: the node-DP model [25, 10], and the shuffle model [18]. We note that our analysis of approximate smooth sensitivity holds for other privacy models as well, and we expect this could be used for improved private queries for other problems. However, the specific technique using diamond sampling for faster triangle counting only holds in the edge-DP model. Fundamentally new ideas are needed for extending this technique to node-DP and others. For a more comprehensive review of recent development on subgraph counting with privacy, we refer readers to [27].

3 Preliminaries

A non-induced embedding of a graph H = ( V H , E H ) into a graph G = ( V, E ) is a mapping ϕ : V H → V such that ( ϕ ( u ) , ϕ ( v )) ∈ E whenever ( u, v ) ∈ E H . We use f H ( G ) to denote the number of non-induced embeddings of a H in G . We drop the subscript H when it is clear from the context. Let ℓ = | V H | and n = | V | . A denotes the adjacency matrix of graph G . Let N ( i ) and N ( i ) denote the set of neighbors and non-neighbors of node i ∈ V , respectively; let d ( i ) = | N ( i ) | and ¯ d ( i ) = n -1 -d ( i ) . Let d max = max v ∈ V d ( v ) denote the maximum degree.

Differential privacy on graphs. Let G denote a set of graphs on a fixed set V of nodes. For a graph G ∈ G , we use V ( G ) and E ( G ) to denote the set of nodes and edges of G , respectively. In this paper, we will focus on the notion of edge privacy [3], where all graphs G ∈ G have a fixed set of nodes V ( G ) = V , and two graphs G,G ′ ∈ G are considered neighbors, i.e., G ∼ G ′ , if they differ in exactly one edge, i.e., | E ( G ) -E ( G ′ ) | = 1 .

Definition 1. A (randomized) algorithm M : G → R is ( ϵ, δ ) -differentially private if for all subsets S ⊂ R of its output space, and for all G,G ′ ∈ G , with G ∼ G ′ , we have Pr [ M ( G ) ∈ S ] ≤ e ϵ Pr [ M ( G ′ ) ∈ S ] + δ [12, 3].

Problem statement: subgraph counting with edge differential privacy. Given a family of graphs G on a set V of vertices, a subgraph H , and parameters ϵ, δ , construct an ( ϵ, δ ) -differentially private mechanism M f H : G → 2 V , such that | M f H ( G ) -f H ( G ) | is minimized.

We discuss the notions of sensitivity mostly using the notation from [23, 31], with slight changes. Let LS f ( G ) = max G ′ ∼ G | f ( G ) -f ( G ′ ) | denote the local sensitivity of f ; we also use LS ( f ( G )) to denote this. GS ( f ) = max G LS ( f ( G )) denotes the global sensitivity. The local sensitivity of a function f on a graph G at distance t is defined as LS ( t ) f ( G ) = max d ( G,G ′ ) ≤ t LS f ( G ′ )

Definition 2. (Smooth bound on LS [31]) For β > 0 , a function S : D n → R + is a β -smooth upper bound on LS f if it satisfies the following conditions: (1) for all x ∈ D n : S ( x ) ≥ LS f ( x ) , and (2) for all x ∼ y ∈ D n : S ( x ) ≤ e β · S ( y ) .

S ∗ f,β ( G ) = max G ′ ( LS ( f ( G ′ )) · e -βd ( G,G ′ ) ) is the smallest function satisfying Definition 2, and is referred to as the β -smooth sensitivity of f at G . Using the local sensitivity at distance t , the smooth sensitivity can be written as S ∗ f,β ( G ) = max t =1 ,..., ( n 2 ) e -tβ LS ( t ) f ( G ) .

Definition 3. (Admissible Noise Distribution) [31] A probability distribution on R d given by a density function h is ( α, β ) -admissible if, for α = α ( ϵ, δ ) , β = β ( ϵ, δ ) , the following conditions hold for all ∆ ∈ R d and λ ∈ R satisfying || ∆ || 1 ≤ α and | λ | ≤ β , and for all measurable S ⊆ R d : (1) Sliding property: Pr Z ∼ h [ Z ∈ S ] ≤ e ϵ/ 2 Pr Z ∼ h [ Z ∈ S +∆]+ δ/ 2 , and (2) Dilation property: Pr Z ∼ h [ Z ∈ S ] ≤ e ϵ/ 2 Pr Z ∼ h [ Z ∈ e λ · S ] + δ/ 2 .

An important example of an admissible noise distribution is the Laplace mechanism Lap ( λ ) probability density function is h ( z ) = 1 2 λ e -| z | /λ .

Lemma 1. (Calibrating noise to Smooth Sensitivity [31]) Let Z be a random variable sampled from an ( α, β ) -admissible noise. Let S f,β be the β -smooth upper bound on the local sensitivity of f . Then algorithm A ( x ) = f ( x ) + S f,β ( x ) α Z is ( ϵ, δ ) -differentially private.

Definition 4. (An ( α, δ ) -approximation to a function f ). ˜ f is said to be an ( α, δ ) -approximation to a function f if for any input x , with probability at least 1 -δ , we have (1 -α ) f ( x ) ≤ ˜ f ( x ) ≤ (1 + α ) f ( x ) .

Social networks generally have the property that nodes have high clustering coefficient, which is the fraction of pairs of neighbors of a node which are connected. We consider a more restricted notion here. We refer to a path with r edges as an r -path. We say a 2-path i, j, k is 'closed' if ( i, k ) ∈ E , i.e., i, j, k form a triangle in G .

Definition 5. ( ( C, Λ) -transitive graph) A graph is said to be ( C, Λ) -transitive if for any vertex j ∈ V and edge ( i, j ) ∈ E , we have: if deg ( j ) > Λ , then C fraction of all the 2-paths starting with i, j are closed.

4 Approximate smooth sensitivity and fast private triangle counting

We first show that we can ensure privacy even if the smooth sensitivity S ∗ f,β is estimated approximately; we then extend it to show that this works when f ( G ) and S ∗ f,β both are estimated approximately.

Definition 6. ˜ S f,β is said to be a γ -upper approximation of the smooth sensitivity S ∗ f,β of function f if S ∗ f,β ( D ) ≤ ˜ S f,β ( D ) ≤ e γ S ∗ f,β ( D ) for any dataset D . ˜ S f,β is said to be a ( γ, δ ′ ) -upper approximation of the smooth sensitivity S ∗ f,β of function f if ˜ S f,β is a γ -upper approximation for any dataset D with probability at least 1 -δ ′ .

We observe below that calibrating noise using a ( γ, δ ′ ) -approximation of the smooth sensitivity gives us privacy. We assume f is a real valued function, since we are focused on graph statistics.

Approximate Smooth Sensitivity for Approximate Query. We now show that approximate smooth sensitivity can be used even when f ( G ) is computed approximately, which becomes a bit more challenging.

Let A f be ( α 1 , δ 1 ) -approximation of a function (query) f for a small constant α 1 < 1 / 2 . Let ˜ S f,β be ( γ, δ 2 ) -upper approximation to S ∗ f,β . We will show that we can utilize A f and ˜ S f,β to calculate a differentially private version of the function f .

For the purpose of privacy analysis, we define the functions g f , s f , and S g f as below. Those functions may not be computed efficiently. However, they are only used for the analysis, and the actual algorithm does not compute them. The actual computation (Algorithm 3) will only utilize A f and ˜ S f,β to output the differentially private version of f . We follow some of the analysiss of [4] to prove our privacy guarantees below. We share (with [4]) the same process of proving that S g f ( D ) is a smooth-upper bound of the local sensitivity (Smooth Sensitivity) of g f . The main difference is our analyses has to take into account the newly defined function s f -which is a bounded variant of the Approximate Smooth Sensitivity ( ˜ S f,β , which approximates S ∗ f,β ), while [4] uses the global sensitivity GS f of f . As we can see in Lemma 13, the second condition requires S g f ( D ) ≤ e β ′ S g f ( D ′ ) for any pair of neighbor datasets D ∼ D ′ . While GS f remains unchanged in both D and D ′ , ˜ S f,β may have different values for D and D ′ which makes it more difficult to analyze S g f . We also have to take into account the small probability δ 2 that ˜ S f,β ( D ) / ∈ { S f,β ( D ) , e γ S f,β ( D ) } , that will add up in the δ -part of ( ϵ, δ ) -DP of the final output.

Definition 7. Let A f be an ( α 1 , δ 1 ) -approximation of f . We define functions g f and s f as:

Let S g f ( D ) = 4 α 1 g f ( D ) + 2 s f ( D )

Lemma 3. (Lemma 13) Given g f , s f , and S g f as defined above, S g f is a β ′ -smooth upper bound of the local sensitivity of g f , where β ′ ≥ 4 α 1 + γ + β and α 1 < 1 / 2 .

In order to prove that S g f is a β ′ -smooth upper bound of the local sensitivity of g f , we have to prove the two conditions: LS g f ( D ) ≤ Sg f ( D ) , and S g f ( D ) ≤ e β ′ S g f ( D ′ ) for any pair of neighbor datasets D ∼ D ′ . The full proof is in Lemma 13 in the SM. Theorem 1 below summarizes our result.

Theorem 1. (Theorem 4) Let A f be a ( α 1 , δ 1 ) -approximation to f , ˜ S f,β be a ( γ, δ 2 ) -upper approximation to S ∗ f,β , for γ = β = 8 ϵ ln(2 /δ ) , α 1 = 4 ϵ ln(2 /δ ) . Then, A ( D ) = A f ( D ) + Lap ( 8 α 1 A f ( D )+4 ˜ S f,β ( D ) ) is ( ϵ, e ϵ/ 2 +1 δ +2 δ 1 +2 δ 2 ) -differentially private.

4.1 Application: fast private triangle counting

In this section, we study the problem of computing f ∆ ( G ) , the number of triangles in a graph G ( E,V ) , by fast approximation of its smooth sensitivity. Recall the definitions of local sensitivity at distance t , denoted by LS ( t ) f ( G ) , defined in Section 3.

We denote the local and smooth sensitivity of f ∆ ( G ) by LS ( t ) ∆ ( G ) and S ∗ ∆ ,β ( G ) . Let a ij = ∑ k ∈ [ n ] A ik A jk denote the number of common neighbors of nodes i and j in a graph and b ij = ∑ k ∈ [ n ] A ik ⊕ A jk denote the number of nodes that are neighbors of i or j but not both. Then it has been shown that LS ( t ) ∆ ( G ) = max i,j c ij ( t ) where c ij ( t ) = min ( a ij + t +min( t,b ij ) 2 , n -2 ) (Claim 3.13 of [31]). Wecan rewrite this as LS ( t ) ∆ ( G ) = min(max i,j ( a ij + b ij 2 + t 2 ) , max i,j a ij + t, n -2) = min(max i,j deg ( i )+ deg ( j ) 2 + t 2 , max i,j a ij + t, n -2) .

The bottleneck in computing LS ( t ) ∆ ( G ) (and S ∗ ∆ ,β ( G ) ) is estimating ˆ a = max i,j a ij . We adapt the diamond sampling technique of [2] for this task. By doing that, we can quickly calculate ̂ LS ( t ) ∆ ( G ) = min(max i,j deg ( i )+ deg ( j ) 2 + t 2 , ˆ a + t, n -2) The core idea in diamond sampling is to find a 'diamond' (a 4-cycle) of the form ( i, o, j, k ) obtained by the intersection of two 'wedges' (a 2-path) ( i, o, j ) and ( i, k, j ) , as shown in Figure 1. Since any pair ( i, j ) is part of a 2 ij diamonds, the probability of selecting such a diamond is proportional to a 2 ij . This idea is formalized in Algorithm 4. Let W be the matrix W ki = deg ( k ) × deg ( i ) for all ( k, i ) ∈ E , and 0 for all other entries.

We first note that if ˆ a = 0 , it must be the case that G has no paths of length ≥ 2 , for if there is a 2-path i, k, j , then a ij ≥ 1 . Therefore, if ˆ a = 0 , we can determine LS ( t ) ∆ ( G ) exactly in time O ( m ) . Therefore, we assume that ˆ a ≥ 1 in the rest of this section.

Even though Algorithm 4 can quickly estimate the quantity max a 2 ij within a specific bound, it can only do so with the number of iterations that is sufficiently large-which its exact threshold is unknown without the access to the dataset and the value of interest ( max a 2 ij ) itself (see Lemma 15). We overcome this issue with the following idea. First, we propose some estimation τ of max a 2 ij and run an instance of Algorithm 4 with the number of iterations computed from τ . Second, we compare the estimation of max a 2 ij returned by Algorithm 4 with τ to determine if we over- or under-estimate τ . Third, we adjust τ accordingly and repeat the process until we find a good value of τ .

We implement Algorithm 1 based on this idea. In the algorithm, we first set τ to its largest possible value ( n 2 ) knowing that we are over-estimating τ ( τ > max a 2 ij ). We note that by over-estimating τ , Algorithm 4 (line 6 ) may not have enough iterations to guarantee the convergence of max x ij to max a 2 ij . Note that in the output of Algorithm 4, for any i, j , x ij is an approximation of a 2 ij . We

determine if we are truly over-estimating τ by applying Corollary 3 (in the SM), comparing the estimation max ij x ij from Algorithm 4 in line 6 to (3 / 2) τ . The proof utilizes an idea that even though max x ij may not converge to max a 2 ij and we may not determine it directly, max x ij can converge to τ once we under-estimate τ (i.e., τ < max a 2 ij ) and that we can check this convergence easily. Corollary 3 states that when max x ij < (3 / 2) τ , max a 2 ij < 3 τ which means we should lower τ . We keep running Algorithm 4 and reducing τ by 3 / 4 in each iteration until the estimation of max x ij lies above (3 / 2) τ . By Corollary 3, we show that when it happens, τ k ≤ max a 2 ij that guarantees the convergence of max x ij to max a 2 ij . Lemma 4 shows that when the algorithm outputs τ k , w.h.p., the true value of interest max a 2 ij is within τ k and 4 τ k .

Lemma 4. (Lemma 18 in SM) Let τ k be the output of Algorithm 1. With probability at least 1 -1 /n c -3 , τ k < max ij a 2 ij < 4 τ k .

Algorithm 2 calculates an approximation of max a 2 ij for the Approximated Smooth Sensitivity with any γ (which is required for Differential Privacy with an arbitrary ϵ ). Since τ k is guaranteed to be close to the true value of max a 2 ij (within a constant factor of 4 ), we run the last instance of Algorithm 4 with some constant θ , which is derived from the target approximation factor γ . Lemma 5 shows that the estimation output by Algorithm 4 in line 6 falls within the range of (1 -θ, 1 + θ ) of the true value. Following that, Theorem 2 guarantees that the quantity output by Algorithm 2 is a ( γ, δ ) -upper approximation to Smooth Sensitivity S ∆ ,β ( G ) of the triangle count of graph G .

Lemma 5. (Lemma 19 in SM) With probability at least 1 -δ in Algorithm 2, max ij x ij ∥ W ∥ s ∈ (1 -θ, 1 + θ ) max ij a 2 ij .

Theorem 2. (Theorem 5, 6 in SM) Suppose max ij a ij > 0 . Algorithm 2 outputs ˜ S ∆ ,β ( G ) , which is a ( γ, δ ) -upper approxmiation to S ∆ ,β ( G ) , in O ( log n ( 1 2 δ ) ∥ W ∥ 1 log m log n ( e 2 γ -1 e 2 γ +1 ) 2 max ij a 2 ij + m + log n β + n ) time.

Proof. (Sketch) Algorithm 1 has a run-time of kO ( s ′ log m )) where s ′ = O ( ∥ W ∥ 1 c log n/ ( θ 2 τ )) as it executes Algorithm 4 with varying s = s ′ and k is the number of search steps. Since at the end of the search τ k > max ij a 2 ij / 4 w.h.p., we know that k = O (log 4 n 2 max ij a 2 ij ) . In the first step, we start with τ 0 = n 2 and reduce τ k by a factor of 3 / 4 after each step. The total running time will be O ( ∥ W ∥ 1 × log m × c log n θ 2 × 1 n 2 (1 + (3 / 4) -1 + . . . (3 / 4) -k ) = O ( ∥ W ∥ 1 c log m log n θ 2 max ij a 2 ij ) . Finally,

( )

Algorithm 2 has a run-time of O log n (2 δ ) ∥ W ∥ 1 log m log n ( e 2 γ -1 e 2 γ +1 ) 2 max ij a 2 ij + m + log n β + n taking into account the time required to compute W ( O ( m ) ) and ˜ S ∆ ,β ( G ) ( O (log ( n ) /β + n ) ) and substituting the values of c and θ as described in the Algorithm.

Corollary 1. Suppose max ij a ij > 0 , G is ( C, Λ) -transitive (as in Definition 5), and δ ≥ n -2 . Then, for any parameter γ > 0 . Algorithm 2 outputs ˜ S ∆ ,β ( G ) , which is a ( γ, δ ) -upper approxmation to S ∆ ,β ( G ) , in O ( m max(1 /C 2 , Λ 2 )) log m log n/ min( γ 4 , 1 / 4) + n ) time.

Proof. Since ( e 2 γ -1 e 2 γ +1 ) 2 > γ 4 for 0 < γ < 1 / √ 2 , we have ( e 2 γ -1 e 2 γ +1 ) 2 > min( γ 4 , 1 / 4) . Next, for any node i and j ∈ N i , we have deg ( j ) max i ′ j ′ a i ′ j ′ ≤ max { 1 C , Λ } : if deg ( j ) > Λ , it must be the case that at least C -fraction of the paths i, j, k for k ∈ N j are closed, which means a ij ≥ C · deg ( j ) , and so deg ( j ) max i ′ j ′ a i ′ j ′ ≤ 1 C ; if deg ( j ) ≤ Λ , we have deg ( j ) max i ′ j ′ a i ′ j ′ ≤ Λ . Therefore, ∥ W ∥ max ij a 2 ij = ∑ i ∈ V ( G ) ∑ j ∈N i deg ( i ) × deg ( j ) max i ′ j ′ a 2 i ′ j ′ = ∑ i ∈ V ( G ) deg ( i ) max i ′ j ′ a i ′ j ′ ∑ j ∈N i deg ( j ) max i ′ j ′ a i ′ j ′ ≤ ∑ i ∈ V ( G ) ∑ j ∈N i max( 1 C 2 , Λ 2 ) = 2 m max( 1 C 2 , Λ 2 ) , the Corollary follows.

5 Higher-order local sensitivity and improved bounds for path counting

As mentioned earlier in Section 2, Karwa et al. [23] develop a different technique involving local sensitivity of the local sensitivity, and use it for private counts of k -cliques, since it seems hard to

"

#

#

!

#

!

#

"

#′

"

#′

!

#′

#

#

$

#′

$

#

%

#′

%

!

#

!

#

"

#

$

#

!

#

"

#

#

#′

!

#′

!

#′

"

#′

"

#

#

#′

#

#′

#

#′

$

#′

%

#

%

#

&

#

%

#′

%

#′

&

estimate smooth sensitivity for such subgraphs. We generalize it, and develop a multi-level local sensitivity approach, extending [23], for privately counting the number of copies of any subgraph H ; as in the case of smooth sensitivity, we show that this also works with approximate queries.

Let ℓ denote the number of edges in H . Our main idea involves finding private bounds for higherorder local sensitivity. Let L = {{ i, j } : i, j ∈ V } denote all possible pairs of nodes in V . For subgraph H , and a subset S ⊂ {{ i, j } : i, j ∈ V } of pairs of nodes, let f H ( G,S ) denote the number of embeddings of H in the graph G ( V, E ∪ S ) , which contain all the edges corresponding to pairs of nodes in S . For S = ∅ , f H ( G, ∅ ) is the number of embeddings of H in G . We drop the subscript H when it is clear from the context. Let f ( k ) ( G ) = max | S | = k f ( G,S ) ; note that f (0) ( G ) = f ( G ) . It can be seen that LS ( f ( k ) ) ≤ f ( k +1) ( G ) , which is the basis for considering higher-order local sensitivity for subgraph counting. However, a careful analysis of the privacy bounds becomes involved. Our approach is described in Algorithm 5 in the supplementary material, and the main result is summarized below.

Theorem 3. Let g ( k ) ( G ) = f ( k ) ( G )+ g ( k +1) ( G ) ln 1 /δ 2 ϵ 2 + Lap ( g ( k +1) ( G ) /ϵ 2 ) , for k = ℓ -1 , · · · , 0 as computed in Algorithm 5. Then, g (0) ( G ) is an (( k m +1) ϵ 2 , δ 2 +( k m +1) e ϵ 2 δ 2 ) -DP estimate of f ( G ) .

The above result, and the definition of f ( k ) ( G ) implies that for any subgraph with ℓ edges, the higher-order local sensitivity approach can be run in time O ( n 2 ℓ ) . This can be improved further for paths and trees, as summarized below.

Lemma 6. Private counting for a graph H with a constant ℓ number of edges using higher-order local sensitivity (Algorithm 5) has running time O ( n 2 ℓ ) . When H is a path or tree with ℓ edges, Algorithm 5 has running time O ( n ℓ +1 ) .

Higher-order local sensitivity with approximate queries. We show that our method works even if we have a very good approximation ˆ f ( x ) to f ( x ) , instead of the exact value (Theorem 7 in the supplementary material). This allows us to use ˆ f ( x ) instead of f ( x ) in Algorithm 5.

Improved bounds for path counting in graphs with bounded degeneracy and treewidth. We say that G has degeneracy d , if the nodes can be ordered so that each node has at most d neighbors with higher index [13]. Informally, G has treewidth t if it has recursive balanced separators of size t ; see [11] for details. There has been a lot of work showing that non-private algorithms for subgraph counting can be done efficiently when G has either of these parameters bounded, e.g., [5, 13, 6, 11]. We show that if G has degeneracy d and treewidth t , then the higher-order local sensitivity of paths of length k can be computed in time O ( h ( d, t, k ) poly ( n )) , where h ( d, t, k ) is independent of n ; thus the running time does not scale as n k .

Our algorithm and analysis are summarized in Section E in the supplementary material (Theorem 8). Recall that the goal is to compute f ( k ) ( G ) = max S : | S |≤ k f ( G,S ) . We summarize the main ideas here. We show that a feasible solution, which involves fixing a subset S of pairs of nodes, can be viewed as a tuple of at most k disjoint subgraphs ( H i , D i ) , referred to as k -compact subgraphs. This is

defined to be a subgraph which satisfies the following conditions (Figure 6): (1) H = G σ ( v, k, F ( i ) ) for some node v and subset F ( i ) = { e 1 , . . . , e i } , where G σ is the k -hop graph starting at node v , only restricted to higher degeneracy order nodes than v , (2) the k -hop neighborhood is expanded as edges in F ( i ) (referred to as internal anchor edges) are added, one at a time, and (3) D is a subset of vertices in H , and is referred to as the set of connectors. The compact subgraphs are connected via edges between connector nodes.

We show that paths using edges from S (which contribute to the count f ( G,S ) ) can be viewed to consist of segments within these compact subgraphs, passing through internal anchor edges, as well as edges between connectors (which are also from S ). We guess the number of path segments between connectors within a factor of (1 + 1 / (log n ) c ) for a constant c , which allows us to search for a tuple of compact subgraphs with the corresponding approximate counts. We show that the number of such tuples is O ( h ( d, k ) poly ( n )) , and existence of given tuple can be determined in a similar time.

6 Experimental Results

We evaluate our algorithm for privately computing #triangles using approximate smooth sensitivity (Algorithm 2), compared with the exact version of smooth sensitivity and other baselines.

Datasets. We consider different real-world networks from [26] as inputs. The networks have have sizes range from 10 K -300 K nodes, with one of the largest ones having over 2 million edges. Statistics of the networks are presented in Table 2. Due to space limit, we only present the results for some selected networks here. The full results are presented in the SM.

Infrastructure. All algorithms are implemented in C++ and OpenMP framework for parallelization. We ran our experiments on a system with a 48-core Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz and 1.5TB RAM and limit the number of parallel threads in all experiments to 40.

Baselines. We implement two baseline methods: the exact smooth sensitivity of triangle count to calibrate Laplacian noise (denoted in our experiment as Karwa et al.) [23], and using the ladder function method (Zhang et al.) [36].

Metrics for evaluating performance. For each input network G , we calculate the following metrics: · Accuracy of the private triangle counts, defined as TRIANGLE COUNT RELATIVE ERROR = | M f ∆ ( G ) -f ∆ ( G ) | f ∆ ( G ) , where M f ∆ is the tested private mechanism to output f ∆ ( G ) .

· Speedup of Algorithm 2, which compares its running time with the time required for [36, 23] SPEEDUP = Runtime of our algorithm on G Runtime of baseline algorithm on G (Figure 5).

· Approximation error of Algorithm 2 for estimating smooth sensitivity, defined as SENSITIVITY APPROXIMATION ERROR = ˜ S β ( G ) -S β ( G ) S β ( G ) (Figure 4).

Parameters. We test our algorithms at different privacy budgets ϵ ∈ { 2 {-3 , -2 , -1 , 0 , 1 } } and a fixed δ = 10 -6 . We report the average accuracy and runtime of five (5) repeats of the private triangle count via approximate smooth sensitivity experiments due to the randomness used in the algorithm. For the exact calculation, we calculate the smooth sensitivity once and re-sample the noise in each run, since

the exact calculation of the sensitivity is deterministic and does not change for each setting. We use the true counts of triangles of the networks as reported by [26].

0.4

Triangle Count Rel. Error

0.3

0.2

0.1

0.0

Oregon-1_G_1

Oregon-2_G_1

0.4

Private

0.3

0.2

0.1

- 0°0

0.125

0.250

0.500

1.000

2.000 0.125

0.250

0.500

1.000

2.00C

epsilon

Alg

wuuobje no

+

Karwa et al.

+

2.0

1.5

Error

1.0

Approx.

0.5

0.0

Sensitivity

Oregon-1_G_1

Oregon-2_G_1

2.0

1.5

1.0

F S'0

00

0.125

0.250

0.500

1.000

2.000 0.125

0.250

0.500

1.000

2.00C

epsilon

Alg  Approx.Error ** Gamma.Upper

Experimental Results. The results show that our algorithm can achieve similar levels of accuracy to the exact calculation while being several orders of magintude faster. Figure 3 shows that when using approximate smooth sensitivity, the private triangle counts reach the accuracy of using the exact calculation of the smooth sensitivity across different privacy budgets in all but two networks (Oregon 1 and Oregon 2). In these two networks, the smooth sensitivities are relatively large in comparison with the triangle counts, which makes the noise fluctuate more for the approximate estimate (see Figure 8).

100 4

10 ·

Oregon-1_G_1

Oregon-2_G_1

100 4

10

0.125

0.250

0.500

1.000

2.000 0.125

epsilon

0.250

0.500

1.000

2.000

Speedup

over Karwa et al.

function (Zhang et al.)) on selected networks. Figure 4 shows that all approximate smooth sensitivities are within some small factor of the exact smooth sensitivities. Noter that approximate smooth sensitivities are always larger than the true smooth sensitivity. It is important since a lower value may expose the privacy due to an inadequate noise magnitude. In general, a smaller value of ϵ (higher privacy guarantee) requires a more accurate estimation of the sensitivity. It is illustrated in Figure 4 as the approximate smooth sensitivity is close to the exact smooth sensitivity in all networks at ϵ = 0 . 125 or ϵ = 0 . 25 .

Figure 5 shows that our approximation algorithm is orders of magnitude faster than the exact algorithm. In large graphs (Amazon, DBLP, Gowalla), the speedup may reach 1 , 000 -fold. Generally, a lower ϵ requires a smaller approximation factor ( γ ), which in turn requires a larger number of iterations to reduce the error in approximation. The majority of tested networks have speedup factors between 10 and 100 -fold across all privacy budgets ϵ .

7 Conclusions and future work

We give significant improvement in the running time for privately counting the number of embeddings of constant size subgraphs in a graph, without using noise based on global sensitivity. Despite a lot of work on private subgraph counting, efficient algorithms were not known for many subgraphs. Our results for triangles show significant benefits of our approach, by improving on the runtime over all prior private algorithms. Our approach of using approximations to sensitivity metrics opens the possibility of using other techniques from graph sampling and sketching to obtain more efficient algorithms. For general subgraphs, our focus here has been on theoretical bounds using multi-level local sensitivity combined with approximate queries. Developing efficient and practical algorithms for these problems is a good direction for future work.

Acknowledgments and Disclosure of Funding

This work was partially supported in part by the following grants: National Science Foundation Grants CCF-1918656 (Expeditions), OAC-1916805 (CINES), IIS-1955797, and CNS-2317193, VDH Grant PV-BII VDH COVID-19 Modeling Program VDH-21-501-0135, DTRA subcontract/ARA S-D0018915-TO-01-UVA, NIH 2R01GM109718-07, CDC MIND cooperative agreement U01CK000589, and the U.S. DOE Exascale Computing Project's (ECP) (17-SC-20-SC) ExaGraph codesign center at Pacific Northwest National Laboratory.

References

Hilal Asi and John C. Duchi. Instance-optimality in differential privacy via approximate inverse sensitivity mechanisms. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc.

Grey Ballard, Tamara G Kolda, Ali Pinar, and C Seshadhri. Diamond sampling for approximate maximum all-pairs dot-product (mad) search. In 2015 IEEE International Conference on Data Mining , pages 11-20. IEEE, 2015.

Jeremiah Blocki, Avrim Blum, Anupam Datta, and Or Sheffet. Differentially private data analysis of social networks via restricted sensitivity. In Proceedings of the 4th Conference on Innovations in Theoretical Computer Science , ITCS '13, page 87-96, New York, NY, USA, 2013. Association for Computing Machinery.

Jeremiah Blocki, Elena Grigorescu, Tamalika Mukherjee, and Samson Zhou. How to make your approximation algorithm private: A black-box differentially-private transformation for tunable approximation algorithms of functions with low sensitivity. arXiv preprint arXiv:2210.03831 , 2022.

Marco Bressan. Faster subgraph counting in sparse graphs. In 14th International Symposium on Parameterized and Exact Computation (IPEC 2019) . Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2019.

Marco Bressan and Marc Roth. Exact and approximate pattern counting in degenerate graphs: New algorithms, hardness results, and complexity dichotomies. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS) , pages 276-285. IEEE, 2022.

Shixi Chen and Shuigeng Zhou. Recursive mechanism: Towards node differential privacy and unrestricted joins. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data , SIGMOD '13, page 653-664, New York, NY, USA, 2013. Association for Computing Machinery.

Vincent Cohen-Addad, Chenglin Fan, Silvio Lattanzi, Slobodan Mitrovic, Ashkan Norouzi-Fard, Nikos Parotsidis, and Jakub M Tarnawski. Near-optimal correlation clustering with privacy. Advances in Neural Information Processing Systems , 35:33702-33715, 2022.

Laxman Dhulipala, Quanquan C Liu, Sofya Raskhodnikova, Jessica Shi, Julian Shun, and Shangdi Yu. Differential privacy from locally adjustable graph algorithms: k-core decomposition, low out-degree ordering, and densest subgraphs. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) , pages 754-765. IEEE, 2022.

Xiaofeng Ding, Xiaodong Zhang, Zhifeng Bao, and Hai Jin. Privacy-preserving triangle counting in large graphs. In Proceedings of the 27th ACM international conference on information and knowledge management , pages 1283-1292, 2018.

Rodney G Downey and Michael R Fellows. Fundamentals of parameterized complexity , volume 4. Springer, 2013.

Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3-4):211-407, 2014.

David Eppstein, Maarten Löffler, and Darren Strash. Listing all maximal cliques in sparse graphs in near-optimal time. arXiv preprint arXiv:1006.5440 , 2010.

Michael Hay, Chao Li, Gerome Miklau, and David Jensen. Accurate estimation of the degree distribution of private networks. In 2009 Ninth IEEE International Conference on Data Mining , pages 169-178, 2009.

Jonathan Hehir, Aleksandra Slavkovic, and Xiaoyue Niu. Consistency of privacy-preserving spectral clustering under the stochastic block model. arXiv preprint arXiv:2105.12615 , 2021.

Jacob Imola, Takao Murakami, and Kamalika Chaudhuri. Locally differentially private analysis of graph statistics. In 30th USENIX Symposium on Security , 2021.

Jacob Imola, Takao Murakami, and Kamalika Chaudhuri. { Communication-Efficient } triangle counting under local differential privacy. In 31st USENIX Security Symposium (USENIX Security 22) , pages 537-554, 2022.

Jacob Imola, Takao Murakami, and Kamalika Chaudhuri. Differentially private subgraph counting in the shuffle model. arXiv preprint arXiv:2205.01429 , 2022.

Shweta Jain and C Seshadhri. A fast and provable method for estimating clique counts using turán's theorem. In Proceedings of the 26th international conference on world wide web , pages 441-449, 2017.

Madhav Jha, C Seshadhri, and Ali Pinar. Path sampling: A fast and provable method for estimating 4-vertex subgraph counts. In Proceedings of the 24th international conference on world wide web , pages 495-505, 2015.

Tianxi Ji, Changqing Luo, Yifan Guo, Jinlong Ji, Weixian Liao, and Pan Li. Differentially private community detection in attributed social networks. In Asian Conference on Machine Learning , pages 16-31. PMLR, 2019.

Tianxi Ji, Changqing Luo, Yifan Guo, Qianlong Wang, Lixing Yu, and Pan Li. Community detection in online social networks: A differentially private and parsimonious approach. IEEE transactions on computational social systems , 7(1):151-163, 2020.

Vishesh Karwa, Sofya Raskhodnikova, Adam Smith, and Grigory Yaroslavtsev. Private analysis of graph structure. ACM Transactions on Database Systems (TODS) , 39(3):1-33, 2014.

Shiva P Kasiviswanathan and Adam Smith. On the'semantics' of differential privacy: A bayesian formulation. Journal of Privacy and Confidentiality , 6(1), 2014.

Shiva Prasad Kasiviswanathan, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Analyzing graphs with node differential privacy. In Proceedings of the 10th Theory of Cryptography Conference on Theory of Cryptography , TCC'13, pages 457-476, Berlin, Heidelberg, 2013. Springer-Verlag.

Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection. http://snap.stanford.edu/data , June 2014.

Yang Li, Michael Purcell, Thierry Rakotoarivelo, David Smith, Thilina Ranbaduge, and Kee Siong Ng. Private graph data release: A survey. ACM Comput. Surv. , jan 2023. Just Accepted.

Yvonne Mülle, Chris Clifton, and Klemens Böhm. Privacy-integrated graph clustering through differential privacy. In EDBT/ICDT Workshops , volume 157, 2015.

Dung Nguyen and Anil Vullikanti. Differentially private densest subgraph detection. In 38th International Conference on Machine Learning (ICML) . PMLR, July 2021.

Hiep H Nguyen, Abdessamad Imine, and Michaël Rusinowitch. Detecting communities under differential privacy. In Proceedings of the 2016 ACM on Workshop on Privacy in the Electronic Society , pages 83-93, 2016.

Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Smooth sensitivity and sampling in private data analysis. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing , pages 75-84, 2007.

Davide Proserpio, Sharon Goldberg, and Frank McSherry. Calibrating data to sensitivity in private data analysis. Proceedings of the VLDB Endowment , 7(8), 2014.

Zhan Qin, Ting Yu, Yin Yang, Issa Khalil, Xiaokui Xiao, and Kui Ren. Generating synthetic decentralized social graphs with local differential privacy. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS) , pages 425-438, 2017.

Vibhor Rastogi, Michael Hay, Gerome Miklau, and Dan Suciu. Relationship privacy: Output perturbation for queries with joins. In Proceedings of the Twenty-Eighth ACM SIGMODSIGACT-SIGART Symposium on Principles of Database Systems , PODS '09, page 107-116, New York, NY, USA, 2009. Association for Computing Machinery.

Jessica Shi, Laxman Dhulipala, and Julian Shun. Parallel clique counting and peeling algorithms. In SIAM Conference on Applied and Computational Discrete Algorithms (ACDA21) , pages 135-146. SIAM, 2021.

Jun Zhang, Graham Cormode, Cecilia M. Procopiuc, Divesh Srivastava, and Xiaokui Xiao. Private release of graph statistics using ladder functions. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data , SIGMOD '15, page 731-745, New York, NY, USA, 2015. Association for Computing Machinery.

A Differential Privacy with high probability

Following [24], we define Indistinguishability and Point-wise Indistinguishability as follows:

Definition 8. Two random variables X,Y having the same support are ( ϵ, δ ) -indistinguishable if for all set O ⊆ Supp ( X ) ,

Definition 9. Two random variables X,Y are point-wise ( ϵ, δ ) -indistinguishable if with probability at least 1 -δ over a drawn from either X or Y , we have:

The following lemma allows us to interpret a mechanism that is ϵ -differentially private with probability at least 1 -δ as ( ϵ, δ ) -differentially private.

Lemma 7. (Lemma 3.3 of [24]) If X,Y are point-wise ( ϵ, δ ) -indistinguishable then they are ( ϵ, δ ) -indistinguishable.

We extend the definition of ( ϵ, δ ) -indistinguishability by allowing some small error probability δ ′ :

Definition 10. Two random variables X,Y having the same Support are ( ϵ, δ, δ ′ ) -indistinguishable if for all set O ⊆ Supp ( X ) , with probability at least 1 -δ ′ over a drawn from either X or Y , we have:

The following lemma allows us to interpret a mechanism that is ( ϵ, δ ) -differentially private with probability at least 1 -δ ′ as ( ϵ, δ + δ ′ ) -differentially private.

Lemma 8. If X,Y are ( ϵ, δ, δ ′ ) -indistinguishability then they are ( ϵ, δ + δ ′ ) -indistinguishable.

Proof. Our proof is insired by [24].

We define the set Bad as follow:

Since X,Y are ( ϵ, δ, δ ′ ) -indistinguishability, Pr[ X ∈ Bad ] < δ ′ . We have:

Similarly for Y , we have Pr[ Y ∈ O ] ≤ e ϵ Pr[ X ∈ O ] + δ + δ ′ and the Lemma follows.

B Missing proofs of Section 4

Approximate Smooth Sensitivity for exact queries

When we can calculate a γ -approximation of the smooth sensitivity S ∗ f,β , i.e., ˜ S f,β is a bounded approximation of S ∗ f,β with probability 1 , we can substitute S ∗ f,β by ˜ S f,β to calibrate the noise directly, although with a bit larger effective constant β ′ = β + γ , which in turn will make the noise a bit larger. Lemma 9 formalizes the equivalence of S ∗ f,β and ˜ S f,β in this case.

Lemma 9. If ˜ S f,β is a γ -approximation of the smooth sensitivity S ∗ f,β , then ˜ S f,β is a β ′ -smooth sensitivity of f at β ′ = β + γ .

Proof. By definition of ˜ S f,β and S ∗ f,β , it follows that ˜ S f,β ( D ) ≥ S ∗ f,β ( D ) ≥ LS f ( D ) for all x .

In the case when we can only calculate a ( γ, δ ′ ) -approximation of the smooth sensitivity, i.e., ˜ S f,β is a γ -approximation of S ∗ f,β with probability at least 1 -δ ′ , Lemma 9 cannot be directly applied. Similar to [31], we use the formalism of admissible noise, to prove we can use ˜ S f,β to calibrate an ( α, β ) -admissible noise with appropriate values of α and β to deliver the privacy guarantee. In Lemma 10, we analyze the output of such a mechanism on an arbitrary pair of neighbors D and D ′ . We condition our analysis on the events of success of ˜ S f,β on both D and D ′ (with probability at least 1 -2 δ ′ ). The failure probability (at most 2 δ ′ ) will then add up to the δ -part of the final ( ϵ, δ ) -DP guarantee.

Lemma 10. Let h be ( α, β ′ ) -admissible noise. Let ˜ S f,β be a ( γ, δ ′ ) -approximation to S ∗ f,β , where β = β ′ -γ . Then, A ( D ) = f ( D ) + ˜ S f,β ( D ) α Z is ( ϵ, e ϵ/ 2 +1 2 δ +2 δ ′ ) -differentially private, where Z ∼ h .

Proof. We prove that with probability at least 1 -2 δ ′ , A ( D ) is ( ϵ, e ϵ/ 2 +1 2 δ ) -differentially private, and by applying Lemma 8 the Lemma follows.

Let E be the event that S ∗ ( D ) ≤ ˜ S f,β ( D ) ≤ e γ S ∗ ( D ) ; we have Pr[ E ] ≥ 1 -δ ′ . Similarly, let E ′ be the event that S ∗ ( D ′ ) ≤ ˜ S f,β ( D ′ ) ≤ e γ S ∗ ( D ′ ) for a fixed neighbor D ′ of D ; we have Pr[ E ′ ] ≥ 1 -δ ′ , and Pr[ E ∪ E ′ ] ≥ 1 -2 δ ′ . We will condition on E ∪ E ′ .

Let S ⊂ R + . Let N ( D ) = ˜ S f,β ( D ) α , and define S 1 = Sf ( D ) N ( D ) , S 2 = S 1 + f ( D ) -f ( D ′ ) N ( D ) = Sf ( D ′ ) N ( D ) , S 3 = S 2 N ( D ) N ( D ′ ) = Sf ( D ′ ) N ( D ′ ) .

We observe that A ( D ) ∈ S if and only if Z ∈ S 1 . Similarly, A ( D ′ ) ∈ S if and only if Z ∈ S 3 . We have

where the first inequality is because of Sliding Property and

which satisfies the condition for the Sliding Property, and the second inequality is because of Dilation Property and

which satisfies the condition for the Dilation Property.

Lemma 11 specializes Lemma 10 by using Laplacian noise. We note that this is a generalization of [31]'s analysis of Laplacian noise. We introduce parameter k to control the magnitude of the Laplacian noise, while [31]'s analysis fixed it to 1 . This design gives us the flexibility to choose the values of α and β given fixed targets of ϵ and δ . When k = 1 / 2 , our design yields the same results as [31].

Lemma 11. Let α = kϵ and β = 2 k 2 ϵ log(2 /δ ) for any constant k > 0 , then Lap (2 k ) is an ( α, β ) -admissible noise.

Proof. For som real-valued random variable Y , let ρ δ ( Y ) be the least solution to Pr[ Y ≤ ρ δ ] > 1 -δ . Let δ ′ = δ/ 2 . Let Z ∼ Lap (2 k ) . It follows that | Z | ∼ Exponential (1 / (2 k )) . We will solve the value of ρ δ ′ ( | Z | ) as: 1 -e -2 k | ρ δ ′ ( | Z | ) | = 1 -δ ′ or ρ δ ′ ( | Z | ) = log(2 /δ ) 2 k . Substituting ρ δ ′ ( | Z | ) to the formula of β , we have:

We first prove the Sliding property of Lap (2 k ) . It is equivalent to prove that ln( h ( z +∆) h ( z ) ) ≤ ϵ/ 2 for any | ∆ | < α and h ( z ) is the PDF of Lap (2 k ) . For the distribution Lap (2 k ) , h ( z ) = 1 4 k e -| z | / (2 k ) . We have:

We next prove the Dilation property of Lap (2 k ) , or prove that ln( e λ h ( e λ z ) h ( z ) ) ≤ ϵ/ 2 with probability at least 1 -δ/ 2 . In the first case, setting λ > 0 , we have h ( e λ z ) < h ( z ) , and that ln( e λ h ( e λ z ) h ( z ) ) ≤ λ ≤ kϵ ρ δ ′ ( | Z | ) . Since | Z | ∼ Exponential (1 / (2 k )) , the median of the distribution of | Z | is ln(2) 1 / (2 k ) = 2 ln(2) k . With δ < 1 , we have δ ′ < 1 / 2 and therefore ρ δ ′ ( | Z | ) ≥ median ( | Z | ) = 2 ln (2) k . It follows that ln( e λ h ( e λ z ) h ( z ) ) ≤ kϵ 2 ln(2) k < ϵ 2 .

In the second case, λ < 0 . We consider the ratio h ( e λ z ) h ( z ) :

We then have ln( e λ h ( e λ z ) h ( z ) ) ≤ λ + | z || λ | 2 k < | z || λ | 2 k . Consider the event G : { z : | z | < ρ δ ′ ( | Z | ) } . Under G , we have | z || λ | 1 2 k ≤ ρ δ ′ ( | Z | ) ϵ ρ δ ′ ( | Z | ) 1 2 k = ϵ 2 . Under h , Pr[ G ] = 1 -δ/ 2 , which completes the proof for the Dilation property of Lap (2 k ) .

Finally, we apply a Laplacian noise with appropriate parameters to provide a concrete mechanism using Approximate Smooth Sensitivity ˜ S f,β to guarantee ( ϵ, δ ) -DP.

Lemma 12. (Lemma 2) Let ˜ S f,β be a ( γ, δ ′ ) -approximation to S ∗ f,β , for γ = β = 16 ϵ ln(2 /δ ) . Then, A ( D ) = f ( D ) + Lap ( 2 ˜ S f,β ( D ) ϵ ) is ( ϵ, e ϵ/ 2 +1 2 δ +2 δ ′ ) -differentially private.

Proof. Set k = 4 , using Lemma 11 with α = kϵ = 4 ϵ , and β ′ = β + γ = 32 ϵ ln(2 /δ ) = 2 k 2 ϵ ln(2 /δ ) . Then, from Lemma 11, the Lap (2 k ) = Lap (8) distribution is ( α, β ′ ) -admissible. By Lemma 10, A ( D ) = f ( D ) + ˜ S f,β ( D ) α Z = f ( D ) + ˜ S f,β ( D ) 4 ϵ Lap (8) = f ( D ) + Lap (2 ˜ S f,β ( D ) /ϵ ) is ( ϵ, e ϵ/ 2 +1 2 δ + 2 δ ′ ) -differentially private and the corollary follows.

Approximate Smooth Sensitivity for Approximate queries

Definition 11. (Definition 7) We define a functions g f and s f as:

Algorithm 3 Fast algorithm using Approximate Smooth Sensitivity for Approximate Query A f Input: G,f,ϵ, δ

Output:

( ϵ, ( e ϵ/ 2 +2) δ ) -differentially private estimaton of query f

1: β := γ := 4 α 1 = ϵ 6 log(2 /δ )

2: δ 1 := δ 2 = δ/ 2

3: Calculate the ( γ, δ 2 ) -upper approximate smooth sensitivity ˜ S f,β ( G )

4: Calculate the ( α 1 , δ 1 ) -approximate f A f ( G )

5: Return A f ( G ) + 8 α 1 A f ( G )+4 ˜ S f,β ( G ) ϵ Lap (1)

Lemma 13 proves that S g f is a β ′ -smooth upper bound of the local sensitivity of g f , i.e., we have to prove the two conditions:

LS g f ( D ) ≤ Sg f ( D ) , and

S g f ( D ) ≤ e β ′ S g f ( D ′ ) for any pair of neighbor datasets D ∼ D ′ ,.

where LS g f ( D ) = max D ′ ∼ D | g f ( D ) -g f ( D ′ ) | or the local sensitivity of g f at D .

Lemma 13. (Lemma 3) Given g f , s f , and S g f as defined above, S g f is a β ′ -smooth upper bound of the local sensitivity of g f , where β ′ ≥ 4 α 1 + γ + β and α 1 < 1 / 2 .

Proof. We now prove the two conditions. We first start with proving S g f is an upper bound on the local sensitivity LS g f of g f :

The first inequality uses Definition 7 and the fifth uses the facts, f ( D ) ≤ A f ( D ) 1 -α 1 ≤ 2 g f ( D ) , and LS f ( D ) ≤ S f,β ( D ) (as S f,β ( D ) is a upper bound on LS f ( D ) ).

For the second condition, we need to prove that for any pair D ∼ D ′ , the values of S g f on them do not differ by more than a multiplicative factor of e β ′ with β ′ = 4 α 1 + γ + β .

We note that s f , g f , and S g f are all hypothetical functions, which we may not calculate directly. We need to connect the statement above ( S g f is a Smooth Sensitivity of g f ) to the functions we can compute ( A f and ˜ S f,β ). Again, we define a hypothetical mechanism A ′ that takes D as its input. By the definition of A ′ and the properties of admissible noise, A ′ is ( ϵ, δ ) -DP. Instead of A ′ , we can only calculate A , by replacing g f and s f by A f and ˜ S f,β respectively. Next, we argue that A and A ′ are, in fact, not too different. Lemma 14 conditions on the probability that A and A ′ are the same and prove that the privacy guarantee of A ′ is the same with of A in such condition. The failure probabilities (of A f and ˜ S f,β in approximation of f and S f ) will add up to the δ -part of the final ( ϵ, δ ) -DP.

Lemma 14. Let h be ( α, β ′ ) -admissible noise with β ′ = 4 α 1 + γ + β . With A f and ˜ S f,β defined as above, Algorithm A ( D ) = A f ( D ) + 4 α 1 A f ( D )+2 ˜ S f,β ( D ) α Z is ( ϵ, e ϵ/ 2 +1 2 δ +2 δ 1 +2 δ 2 ) -differentially private, where Z ∼ h .

Proof. We prove that with probability at least 1 -2 δ 1 -2 δ 2 , A ( x ) is ( ϵ, e ϵ/ 2 +1 2 δ ) -differentially private, and by applying Lemma 8, the Lemma follows.

Let A ′ ( D ) = g f ( D ) + S g f α Z = g f ( D ) + 4 α 1 s f ( D )+2 s f ( D ) α Z , as we replace A f and ˜ S f,β by g f and s f respectively. We note that the use of g f and s f is only for the purpose of privacy bound analysis. With β ′ > 4 α 1 + γ + β , Lemma 3 shows that S g f is a β ′ -smooth upper bound on the sensitivity of g f . By Lemma 10, A ′ ( D ) = g f ( D ) + S g f α Z is ( ϵ, e ϵ/ 2 +1 2 δ ) -differentially private.

Since with probability at least 1 -δ 1 , A f ( D ) is g f ( D ) , and at least 1 -δ 2 , ˜ S f,β ( D ) is s f ( D ) , with probability at least 1 -δ 1 -δ 2 , A ( D ) has the same components as A ′ ( D ) , hence they are identical. For a fixed neighbor D ′ ∼ D , we also have that A ′ ( D ) is identical with A ′ ( D ′ ) with probability at least 1 -δ 1 -δ 2 . By Lemma 8, A ( D ) , A ( D ′ ) are ( ϵ, e ϵ/ 2 +1 2 δ, 2 δ 1 +2 δ 2 ) -indistinguishable. It follows that A ( D ) is ( ϵ, e ϵ/ 2 +1 2 δ +2 δ 1 +2 δ 2 ) -differentially private.

Theorem 4. (Theorem 1) Let A f be a ( α 1 , δ 1 ) approximation to f , ˜ S f,β be a ( γ, δ 2 ) -upper approximation to S ∗ f,β , for γ = β = 8 ϵ ln(2 /δ ) , α 1 = 4 ϵ ln(2 /δ ) . Then, A ( D ) = A f ( D ) + Lap ( 8 α 1 A f ( D )+4 ˜ S f,β ( D ) ϵ ) is ( ϵ, e ϵ/ 2 +1 2 δ +2 δ 1 +2 δ 2 ) -differentially private.

Proof. Set k = 4 , using Lemma 11 with α = kϵ = 4 ϵ , and β ′ = β + γ + 4 α 1 = 32 ϵ ln(2 /δ ) = 2 k 2 ϵ ln(2 /δ ) . Then, from Lemma 11, the Lap (2 k ) = Lap (8) distribution is ( α, β ′ ) -admissible. By Lemma 14, A ( D ) = f ( D ) + ˜ S f,β ( D ) α Z = f ( D ) + ˜ S f,β ( D ) 4 ϵ Lap (8) = f ( D ) + Lap ( 2 ˜ S f,β ( D ) ϵ ) is ( ϵ, e ϵ/ 2 +1 2 δ +2 δ 1 +2 δ 2 ) -differentially private and the Theorem follows.

C Missing proofs of Section 4.1

The main target of this section is to calculate an approximation of max a 2 ij , such that we can apply it to approximate a γ -approximation of S ∆ ,β for any γ and β . There are three key components of the analysis:

Algorithm 4 uses diamond sampling to quickly calculate x ij that approximates a 2 ij for every i, j . We use this algorithm as an important sub-routine in Algorithm 1 and 2. Lemma15 shows the utility of the algorithm. We note that due to the Lemma, the quality of each estimation x ij depends on the true value a 2 ij itself-which is unknown and is the value we are trying to get. Hence, we cannot directly apply Algorithm 4 to get any-approximation of max a 2 ij as we need.

Algorithm 1 uses multiple rounds of Algorithm 4 with increasing numbers of iterations. The main idea behind this process is that, with a small number of iterations, Algorithm 4 executes very quickly and gives us a rough estimation x ij of a 2 ij for every i, j . We then check if the estimation is good enough. We know the estimation is considered good when max ij x ij < 3 2 τ k × {some constants} . Corollary 2, 3 will explain why. Finally, Lemma 18 concludes the utility of the output of the algorithm, such that after the last iteration k , τ k is at most 4 times the value of max a 2 ij .

Algorithm 2 further improves the output of Algorithm 1. Now we know roughly how large max 2 ij is, but only within a factor of 4 . We will run the final instance of Algorithm 4 with a special constant θ , which we use to control the approximation of the final estimation. θ is set by γ -the approximation level required for γ -approximation of the smooth sensitivity described by the previous section (which in turn is set by ( ϵ, δ ) ). Lemma 19 concludes the final utility of our estimation.

We define the norm of matrix A as follows: ∥ A ∥ = ∥ A ∥ 1 = ∑ ij A ij , ∥ A i ∥ = ∑ k A ik . Note that degree d ( i ) = ∥ A i ∥ , ¯ d ( i ) = ∥ 1 -A i ∥ . The following concentration bound is shown in [2] for the output of Algorithm 4.

Lemma 15. Lemma 3 of [2] . Fix θ > 0 and error probability β . For any a ij > 0 , if the number of samples s > 3 ∥ W ∥ log(2 /β ) / ( θ 2 a 2 ij ) then

The following two lemmas and corollaries relate upper and lower bounds on the random variable max ij x ij output by Algorithm 4 to the quantity of interest, max ij a 2 ij .

Algorithm 4 Fast estimation of ( a ij ) 2

Input:

G

(

V, E

)

,

Number of iterations

s

Output: x ij that approximates a 2 ij , for all i, j

1: Let A be the adjacency matrix of G

2: for ( k, i ) ∈ E do

3: W ki = d ( k ) d ( i )

4: end for

5: x ← 0

6: for l = 1 , . . . , s do

7: Sample ( k, i ) with probability W ki / ∥ W ∥ 1

8: Sample j from N k

9: Sample o from N i

10: x ij + = A oj

11: end for

12: Return x

Lemma 16 states that if we choose a constant τ that is close but larger than max a 2 ij , then after running Algorithm 4 with s iterations, the maximum of x ij is not larger than τ × {some constants} . Similarly, Lemma 17 states that if we choose a constant τ that is close but smaller than max a 2 ij , then after running Algorithm 4 with s iterations, the maximum of x ij is not smaller than τ × {some constants} .

With appropriate selections of the constants, Corollary 2 and 2 summarize the two lemmas and describe the condition in which we may find a good value for τ . They allow us to observe the output of an instance of Algorithm 4 and compare the outcome with a pre-determined value of τ . Depending on the sign of the comparision max x ij ∥ W ∥ s ≷ τ , we can decide that if we are over-estimating τ (by Corollary 3) or under-estimating τ (by Corollary 2).

The two lemmas and their corollaries below form the basis for Algorithm 1 that finds an appropriate value of τ using binary search.

Lemma 16. For any τ such that max ij a 2 ij < λτ for some constant 0 < λ , there exists a constant λ ′ > λ such that with probability at least 1 -1 poly ( n ) , max ij x ij ∥ W ∥ s ≤ λ ′ τ .

Proof. Consider any i, j . By Chernoff's bound,

As E [ x ij ] = a 2 ij s ∥ W ∥ , we have that

Setting λ ′ = (1+ θ ′ ) a 2 ij τ , we have:

Therefore, we can lower bound θ ′ as follows:

Now, we substitute (1 + θ ′ ) a 2 ij by λ ′ τ :

Selecting λ ′ > λ s.t c ( λ ′ -λ ) 2 λθ 2 ≥ 3 and taking the union bound on all pairs ij , the Lemma follows.

Corollary 2. With c > 3 , θ = 1 2 , λ = 1 , λ ′ = 3 2 , with probability at least 1 -1 /n c -2 , if max ij x ij ∥ W ∥ s > 3 2 τ then max ij a 2 ij > τ.

Lemma 17. For any τ such that max ij a 2 ij > λτ for some constant 0 < λ , then there exists a constant λ ′′ ≤ (1 -θ ) λ such that with probability at least 1 -1 poly ( n ) , max ij x ij ∥ W ∥ s ≥ λ ′′ τ .

Proof. Since max ij a 2 ij > λτ , there exists pairs of nodes i ′ j ′ such that a 2 i ′ j ′ > λτ . For such pairs i ′ j ′ , we have:

Setting λ ′′ ≤ (1 -θ ) λ , we have λ ′′ ≤ (1 -θ ) a 2 i ′ j ′ τ , or λ ′′ τ ≤ (1 -θ ) a 2 i ′ j ′ . Therefore Pr [ x 2 i ′ j ′ ∥ W ∥ s < λ ′′ τ ] ≤ Pr [ x 2 i ′ j ′ ∥ W ∥ s < (1 -θ ) a 2 i ′ j ′ ] ≤ n -cλ . Choosing c such that cλ > 3 and taking the union bound over all pairs i ′ j ′ , the Lemma follows.

Corollary 3. With c > 3 , θ = 1 2 , λ = 3 , λ ′′ = 3 2 , with probability at least 1 -1 /n c -2 , if max ij x ij ∥ W ∥ s < 3 2 τ then max ij a 2 ij < 3 τ.

Lemma 18. (Lemma 4) Let τ k is the output of Algorithm 1, with probability at least 1 -1 /n c -3 , τ k < max ij a 2 ij < 4 τ k .

Proof. Assume the Algorithm 1 stops at step k . At step k -1 , max ij x ij < 3 2 τ k -1 , then by Corollary 3 we have max ij a 2 ij < 3 τ k -1 = 3 4 3 τ k = 4 τ k . At step k , max ij x ij > 3 2 τ k , then by

Corollary 2 we have max ij a 2 ij > τ k . Applying union bound on k steps, with probability at least 1 -1 /n c -3 we have τ k < max ij a 2 ij < 4 τ k , the Lemma follows.

Finally, in Algorithm 2, we postprocess the output of Algorithm 1 to obtain an accurate estimate of max ij a 2 ij and hence, the smooth sensitivity ˜ S ∆ ,β ( G ) .

Lemma 19. (Lemma 5) With probability at least 1 -δ in Algorithm 2, max ij x ij ∥ W ∥ s ∈ (1 -θ, 1 + θ ) max ij a 2 ij .

Proof. We split all pairs ij into 2 sets { i ′ j ′ : a 2 i ′ j ′ < τ } and the remaining pairs { ˜ i ˜ j } that a 2 ˜ i ˜ j > τ . We will prove that the Algorithm will not output x i ′ j ′ for any i ′ j ′ .

For every i ′ j ′ such that a 2 i ′ j ′ < τ , by Lemma 16, set λ = 1 , λ ′ = 3 2 , since c ≥ 12 θ 2 or c ( λ -λ ′ ) λθ 2 = 12 θ 2 1 / 4 θ 2 = 3 , we have max i ′ j ′ x i ′ j ′ ∥ W ∥ s < 3 2 τ with probability at least 1 -1 /n c -2 .

Because max ij a 2 ij > τ k = 3 τ , by Lemma 17, set λ = 3 , λ ′ = 3 2 , since c ≥ 2 or cλ > 3 , we have max ij x ij ∥ W ∥ s > 3 2 τ with probability at least 1 -1 /n c -2 .

Then with probability at least 1 -1 / 2 n c -2 , Algorithm 2 will not output any x i ′ j ′ .

The remaining sets { ˜ i ˜ j } that a 2 ˜ i ˜ j > τ . By Lemma 15, ∀ ˜ i ˜ j : x ˜ i ˜ j ∥ W ∥ s ∈ (1 -θ, 1 + θ ) a 2 ˜ i ˜ j . Applying the union bound on all pair ˜ i ˜ j and setting c ≥ log n (1 / (2 δ )) + 2 , the Lemma follows.

Theorem 5. (Theorem 2) ˜ S ∆ ,β ( G ) output by Algorithm 2 is a ( γ, δ ) -upper approxmiation to S ∆ ,β ( G ) .

Proof. From the result of Lemma 19, we have:

.

Multipling all terms by 1 / (1 -θ ) , we have:

Therefore with probability at least 1 -δ , we have LS ( t ) ∆ ( G ) ≤ ̂ LS ( t ) ∆ ( G ) ≤ e γ LS ( t ) ∆ ( G ) and hence:

with probability at least 1 -δ . It follows that ˜ S ∆ ,β ( G ) is ( γ, δ ) -upper approximation to S ∆ ,β ( G ) .

Theorem 6. (Theorem 2) Suppose max ij a ij > 0 . Algorithm 2 has a runtime of O ( log n (2 δ ) ∥ W ∥ 1 log m log n ( e 2 γ -1 e 2 γ +1 ) 2 max ij a 2 ij + m + log n β + n ) .

Proof. Let m denote the number of edges in G . Then, computing W and ∥ W ∥ takes O ( m ) time and each sampling step requires O (log m ) time. However, we can perform the calculation of W and ∥ W ∥ once and reuse them for all Algorithms (4, 1, and 2). Each sampling step in Algorithm 4 takes O (log m ) time and there are s iterations which results in the total cost of O ( s log m ) .

Algorithm 1 has a run-time of kO ( s ′ log m )) where s ′ = O ( ∥ W ∥ 1 c log n/ ( θ 2 τ )) as it executes Algorithm 4 with varying s = s ′ and k is the number of search steps. Since at the end of the search τ k > max ij a 2 ij / 4 w.h.p., we know that k = O (log 4 n 2 max ij a 2 ij ) . In the first step, we start with τ 0 = n 2 and reduce τ k by a factor of 3 / 4 after each step. The total running time will be O ( ∥ W ∥ 1 log m × c log n θ 2 × 1 n 2 (1 + (3 / 4) -1 + . . . (3 / 4) -k ) = O ( ∥ W ∥ 1 c log m log n θ 2 max ij a 2 ij ) .

account the time required to compute W ( O ( m ) ) and ˜ S ∆ ,β ( G ) ( O (log ( n ) /β + n ) ) and substituting the values of c and θ as described in the Algorithm.

D Details from Section 5

As mentioned earlier, Karwa et al. [23] introduce the notion of local sensitivity of local sensitivity for private counting of k-cliques. In this work, we generalize this notion to multi-level local sensitivity. Recall that f ( G,S ) denotes the number of embeddings of a fixed subgraph H in G that contains all pairs in S and f ( k ) ( G ) = max | S | = k f ( G,S ) . The following lemma is at the core of our approach as it relates local sensitivity to subgraph counting.

Lemma 20. For all k ≥ 0 we have LS ( f ( k ) ) ≤ f ( k +1) ( G ) .

When k ≥ 1 , we have LS ( f ( k ) ) = max { i,j } | f ( k ) ( G ∪ ( i, j )) -f ( k ) ( G -( i, j )) | . We argue below that for each { i, j } ∈ L , we have | f ( k ) ( G ∪ ( i, j )) -f ( k ) ( G -( i, j )) | ≤ f ( k +1) ( G ) , and the statement follows. We have two cases.

Case 1. ( i, j ) ̸∈ G : let S ⊂ L , | S | = k be a subset such that f ( k ) ( G ∪ ( i, j )) = f ( G ∪ ( i, j ) , S ) . Let T ⊂ L, | T | = k be a subset such that f ( k ) ( G ) = f ( G,T ) . By definition of T , we have f ( G,S ) ≤ f ( G,T ) . We have two sub-cases, depending on whether { i, j } ̸∈ S or { i, j } ∈ S . First, suppose { i, j } ̸∈ S . This implies f ( G ∪ ( i, j ) , S ) = f ( G,S ) + f ( G,S ∪ { i, j } ) , and

Next, suppose { i, j } ∈ S . Then, f ( G,S ) = f ( G ∪ ( i, j ) , S ) ≥ f ( G ∪ ( i, j ) , T ) , from the definition of S . Since f ( · , T ) is monotone with respect to the set of edges in the graph, we have f ( G ∪ ( i, j ) , T ) ≥ f ( G,T ) , which implies f ( G,S ) ≥ f ( G,T ) . Combined with the earlier observation that f ( G,T ) ≥ f ( G,S ) , we have f ( G,T ) = f ( G,S ) . This implies

Thus, when ( i, j ) ̸∈ G , in either sub-case, we have | f ( k ) ( G ∪ ( i, j )) -f ( k ) ( G ) | ≤ f ( k +1) ( G ) .

Case 2. ( i, j ) ∈ G : we have | f ( k ) ( G ∪ ( i, j )) -f ( k ) ( G -( i, j )) | = | f ( k ) ( G ) -f ( k ) ( G -( i, j )) | = | f ( k ) ( G ′ ∪ ( i, j )) -f ( k ) ( G ′ ) | , where G ′ = G -( i, j ) . Repeating the argument in case 1 for the graph G ′ , we have | f ( k ) ( G ′ ∪ ( i, j )) -f ( k ) ( G ′ ) | ≤ f ( k +1) ( G ′ ) ≤ f ( k +1) ( G ) .

Lemma 21. (Lemma 4.4 of [23]) Let B be an ( ϵ 1 , δ 1 ) -differentially private algorithm such that Pr[ B ( x ) ≥ LS f ( x )) > 1 -δ 2 for all x . Consider the algorithm A that runs B ( x ) to obtain

an estimate ˜ LS of the local sensitivity and release both ˜ LS and a noisy estimate f : A ( x ) = ( ˜ LS,f ( x ) + Lap ( ˜ LS/ϵ 2 )) , where ˜ LS = B ( x ) , and where Lap ( λ ) is a Laplace random variable with mean 0 and scale parameter λ . Then A is ( ϵ 1 + ϵ 2 , δ 1 + e ϵ 1 δ 2 ) -DP.

Algorithm 5 Estimating higher-order private local sensitivity.

Input:

G,ϵ, δ , subgraph H with ℓ edges

Output:

Private estimate g (0) for f ( G )

1: Let k m = ℓ

2: for k = k m -1 down to 0 do

3: g ( k ) ( G ) = f ( k ) ( G ) + g ( k +1) ( G ) ln 1 /δ 2 ϵ 2 + Lap ( g ( k +1) ( G ) /ϵ 2 )

4: end for

5: return g (0) ( G )

We now give the proof of correctness for Algorithm 5 using Theorem 3.

Proof of Theorem 3

Proof. We prove by induction on k ≥ 1 that g ( k m -k ) ( G ) computed by Algorithm 5 is a (( k + 1) ϵ 2 , δ 2 +( k +1) e ϵ 2 δ 2 ) -DP private estimate, and Pr[ g ( k m -k ) ( G ) ≥ f ( k ) ( G )] ≥ 1 -δ 2 .

The base case is k = 1 . Note that for k m = ℓ , f ( k m ) ≤ 1 , since once ℓ edges are fixed, either we get a fixed subgraph, or it doesn't result in an embedding. Since LS ( f ( k m -1) ) ≤ f ( k m ) , it follows that g ( k m -1) is an ( ϵ 2 , 0) -DP estimate. Further, Pr[ g ( k m -1) < f ( k m -1) ] = Pr[ Lap ( f ( k m ) /ϵ 2 ] < -f ( k m ) ln(1 /δ 2 ) /ϵ 2 ] < δ 2 , which proves the base case.

Next, consider k > 1 . From Lemma 20, LS ( f ( k m -k ) ) ≤ f ( k m -k +1) . Since g ( k m -k +1) is an ( kϵ 2 , δ 2 + ke ϵ 2 δ 2 ) -DP estimate, and Pr[ g ( k m -k +1) > f ( k m -k +1) ] > 1 -δ 2 (by induction), it follows from Lemma 21 that g ( k m -k ) is an ( kϵ 2 + ϵ 2 , δ 2 + ke ϵ 2 δ 2 + e ϵ 2 δ 2 ) -DP estimate. Also, Pr[ g ( k m -k ) > f ( k m -k ) ] = Pr[ Lap ( g ( k m -k +1) /ϵ 2 ) > -g ( k m -k +1) ln 1 /δ 2 /ϵ 2 ] > 1 -δ 2 , and the inductive step follows.

Finally, we analyze the running time of Algorithm 5 using Lemma 6.

Proof of Lemma 6

Proof. Observe that f ( G,S ) can be computed in time O ( n 2( ℓ -| S | ) , so that f ( k ) can be estimated in time O (( n 2 ) ℓ ) . Therefore, the total running time for Algorithm 5 is O ( n 2 ℓ ) .

Improved time for paths of length ℓ . We show that for paths with ℓ edges, the running time can be actually improved from O ( n 2 ℓ ) to O ( n ℓ +1 ) , by keeping track of slightly more complicated information. For a subset S ⊂ L of pairs of nodes, let h ( S ) denote the set of unique nodes in S . Let π be a function, such that for each u ∈ h ( S ) , π ( u ) specifies the position node u has in the embedding of the path. Let f ( G,S,π ) the number of paths containing the edges in S , but with the additional constraint that the ordering with respect to π be satisfied. Then, we have f ( k ) ( G ) = max | S | = k,π f ( G,S,π ) .

Proof. (Sketch) Observe that f ( G,S,π ) can be computed in time O ( n ℓ +1 -| h ( S ) | ) time, since this involves considering the nodes in the remaining positions of the path, other than those specified by π . Further, f ( k ) ( G ) can be computed in O ( n ℓ +1 ) time, since we can first guess the set h ( S ) and the ordering π (instead of directly guessing | S | pairs of nodes), and then guess the remaining nodes.

Theorem 7. Let B be an ( ϵ 1 , δ 1 ) -differentially private algorithm such that Pr[ B ( x ) ≥ LS ( f, x )] > 1 -δ 2 for all x . Let ˆ f ( x ) ∈ [(1 -ϵ 3 ) f ( x ) , (1 + ϵ 3 ) f ( x )] be an approximation to f ( x ) , with ϵ 3 small enough so that ϵ 3 f ( x ) ≤ LS ( f, x ) for all x . Consider the algorithm A that runs B ( x ) releases: A ( x ) = ( ˜ LS, ˆ f ( x ) + Lap ( ˜ LS/ϵ 2 )) , where ˜ LS = B ( x ) , and where Lap ( λ ) is a Laplace random variable with mean 0 and scale parameter λ . Then A is ( ϵ 1 +(3 + ϵ 3 ) ϵ 2 , e ϵ 1 ( δ 1 + δ 2 ) + δ 2 ) -DP.

Proof. The proof is a modification of Lemma 4.4 of [23]). Define the following events

Let P x , P y and P mix correspond to the distributions of these events.

We first compare P y ( S ) and P mix ( S ) for any subset S . Observe that P mix ( S ) P y ( S ) = Pr[ B ( x ) ∈ S ] Pr[ B ( y ) ∈ S ] . Since B is ( ϵ 1 , δ 1 ) -DP, we have Pr[ B ( y ) ∈ S ] ≤ e ϵ 1 Pr[ B ( x ) ∈ S ] + δ 1 , which implies P y ( S ) ≤ e ϵ 1 P mix ( S ) + δ 1 .

Next, we compare P mix ( S ) and P y ( S ) for any subset S . Let F denote the event B ( x ) = ˜ LS ( f, x ) ≥ LS ( f, x ) . Then, P x ( S | F ) P mix ( S | F ) = Pr[ Lap ( z/ϵ 2 ) ∈ S -ˆ f ( x )] Pr[ Lap ( z/ϵ 2 ) ∈ S -ˆ f ( y )] ≥ exp ( -| ˆ f ( y ) -ˆ f ( x ) | z ϵ 2 ) , where z = ˜ LS ( f, x ) . We have that the fraction | ˆ f ( y ) -ˆ f ( x ) |

z

where the second inequality holds because | f ( x ) -f ( y ) | ≤ LS ( f, x ) ≤ ˜ LS ( f, x ) , with probability 1 -δ 1 .

This implies P x ( S | F ) P mix ( S | F ) ≥ e -(3+ ϵ 3 ) ϵ 2 with probability 1 -δ 1 . Equivalently, P mix ( S | F ) ≤ e (3+ ϵ 3 ) ϵ 2 P x ( S | F ) + δ 1 . Therefore, P mix ( S ) = P mix ( S ∧ F ) + P mix ( S ∧ ¯ F ) ≤ e (3+ ϵ 3 ) ϵ 2 P x ( S ) + δ 1 + δ 2 . Putting everything together, P y ( S ) ≤ e ϵ 1 P mix ( S ) + δ 1 ≤ e ϵ 1 +(3+ ϵ 3 ) ϵ 2 P x ( S ) + e ϵ 1 ( δ 1 + δ 2 ) + δ 2 .

E Estimating higher-order local sensitivity for path counts

Let H denote paths of length ℓ . We assume the graph G has degeneracy d and treewidth t . We show that f ( k ) ( G ) = max | S | = k f H ( G,S ) can be computed in time O ( poly ( n )( dt ) k ) under these assumptions on G (i.e., in time smaller than O ( n 2 ℓ ) ). Let σ denote a degeneracy ordering of the nodes in V . So node v has index σ ( v ) . Let N σ ( v ) = { v ′ : v ′ ∈ N ( v ) , σ ( v ′ ) > σ ( v ) } be the set of neighbors of v with index larger than σ ( v ) ; by definition of degeneracy, it follows that | N σ ( v ) | ≤ d for all v . Let v, u be two nodes such that σ ( v ) < σ ( u ) . Then, dist ( v, u, σ, K ) denotes the length of the shortest path from v to u in a graph K , when restricted to nodes v ′ with σ ( v ) < σ ( v ′ ) < σ ( u ) .

Let G ( ˆ E ) denote the graph induced by E ∪ ˆ E . Let N σ ( v, k, ˆ E ) = { u : dist ( v, u, σ, G ( ˆ E )) ≤ k } be the set of nodes within k -hops of v in the graph G ( ˆ E ) , restricted to higher index nodes than v . Let G σ ( v, k, ˆ E ) denote the graph induced by the subset N σ ( v, k, ˆ E ) of nodes. We say a pair e = ( u ′ , v ′ ) is an internal anchor pair in G σ ( v, k, ˆ E ) , if u ′ , v ′ ∈ N σ ( v, k, ˆ E ) , and ( u ′ , v ′ ) ̸∈ E ( G σ ( v, k, ˆ E )) (i.e., ( u ′ , v ′ ) is a newly added edge); else, we say ( u ′ , v ′ ) is an external anchor pair with respect to G σ ( v, k, ˆ E ) . We will add internal anchor pairs sequentially. Let e i be the edge added in the i th step, and let F ( i ) = { e 1 , . . . , e i } .

k -compact subgraph. We define ( H,D ) to be a k -compact subgraph if

H = G σ ( v, k, F ( i ) ) for some node v and subset F ( i ) = { e 1 , . . . , e i

i ≤ k

"

#

#

!

#

!

#

"

#′

"

#′

!

#′

#

#

$

#′

$

#

%

#′

%

!

#

!

#

"

#

$

#

!

#

"

#

#

#′

!

#′

!

#′

"

#′

"

#

#

#′

#

#′

#

#′

$

#′

%

#

%

#

&

#

%

#′

%

#′

&

The pair e 1 = ( u 1 , v 1 ) is an internal anchor pair in G σ ( v, k, ∅ ) . For each j = 2 , . . . , i , the pair e j = ( u j , v j ) is an internal anchor pair in the graph G σ ( v, k, F ( j -1) = { e 1 , . . . , e j -1 } ) .

D is a subset of vertices in H , and is referred to as the set of connectors.

Lemma 22. Let ( H,D ) be a k -compact subgraph. Then H has O ( kd k ) nodes and edges. and the tuple ( H,D ) can be described using O ( k 2 d k +1 log d ) bits.

Proof. (Sketch) We prove by induction on i that G σ ( v, k, F ( i ) ) has at most ( i + 1) d k nodes and edges.

First observe that N σ ( v, k, ∅ ) has at most d k nodes and edges, since that all have index more than σ ( v ) . We have i = 1 as the base case. Let σ ( u 1 ) < σ ( v 1 ) . Then G σ ( v, k, { e 1 } ) will have at most 2 d k more nodes, with at most d k additional reachable nodes through v 1 . In the inductive step, G σ ( v, k, F ( i ) = { e 1 , . . . , e i } ) has at most d k nodes (through v i , assuming σ ( u i ) < σ ( v i ) ) in addition to those in G σ ( v, k, F ( i -1) ) .

The neighborhood of each node u ∈ N σ ( v, k, F ( i ) ) can be represented in O ( d log( kd k )) bits. Therefore, the total number of bits needed to specify H is O ( kd k d log( kd k )) = O ( kd k +1 log( kd k )) . D consists of at most k nodes from N σ ( v, k, F ( i ) ) ; therefore, D can be specified in O ( k log( kd k )) bits. Therefore, ( H,D ) can be specified in O ( kd k +1 log( kd k )) bits, which implies the number of distinct ( H,D ) pairs is at most exp ( kd k +1 log( kd k )) .

Pairing of connector set. The connector set D = { r 1 , . . . , r ℓ } , with ℓ ≤ k , represents nodes which have incident edges outside the compact subgraph H . We might have to consider three kinds of paths, corresponding to Figure 6

As in Figure 6 (Left), these start at a node s in H , then pass through multiple anchor edges, and end at a node t in H . We partition D into two singletons and multiple pairs π 2 ( D ) = { ( r π (1) ) , ( r π (2) , r π (3) ) , . . . , ( r π ( ℓ ) ) } . This corresponds to paths which leave H at node r π (1) and finally return at node r π ( ℓ ) , with disjoint segments from r π (2 j ) to r π (2 j +1) lying within H , for each j . We refer to the set of such pairings as Π 2 ( D ) .

This corresponds to paths which start at a node s in H , then pass through multiple anchor edges, and don't return to a node in H eventually (as in Figure 6 (Middle)). We partition D into one singleton and multiple pairs π 1 ( D ) = { ( r π (1) ) , ( r π (2) , r π (3) ) , . . . , ( r π ( ℓ -1) , r π ( ℓ ) ) } . We refer to the set of such pairings as Π 1 ( D ) .

This corresponds to paths which only pass through nodes in H (as in Figure 6 (Right)). We partition D into multiple pairs π 0 ( D ) = { ( r π (1) , r π (2) ) , . . . , ( r π ( ℓ -1) , r π ( ℓ ) ) } . We refer to the set of such pairings as Π 0 ( D ) .

Approximate counts corresponding to pairings. Note that for a given compact subgraph ( H,D ) , there can be multiple pairings in Π 0 ( D ) , Π 1 ( D ) , Π 2 ( D ) . For each such pairing, e.g., π ∈ Π 0 ( D ) = { ( r π (1) , r π (2) ) , . . . , ( r π ( ℓ -1) , r π ( ℓ ) ) } , we can guess a set of tuples of numbers Num ( π, ˆ E ) = { (( Num ( r π (2 j +1) , r π (2 j +2) ) , j = 0 , 1 . . . )) } , where

Num ( r π (2 j +1) , r π (2 j +2) ) corresponds to the number of segments from r π (2 j +1) to r π (2 j +2) , which are within H , and use some edges from ˆ E as internal anchor edges

The path segments counted in Num ( r π (2 j +1) , r π (2 j +2) ) and those in Num ( r π (2 j ′ +1) , r π (2 j ′ +2) ) are disjoint. In particular, this means that the internal anchor edges used in these counts are disjoint.

The paths within a tuple (( Num ( r π (2 j +1) , r π (2 j +2) ) , j = 0 , 1 . . . )) contain all the internal anchor edges in ˆ E .

In order to reduce the information complexity, we only keep track of Num ( r π (2 j +1) , r π (2 j +2) ) approximately as powers of (1 + µ ) for a small µ ∈ (0 , 1) ; we denote these approximate counts as ̂ Num ( r π (2 j +1) , r π (2 j +2) ) , and the corresponding tuple of counts by ( ̂ Num ( r π (1) , r π (2) ) , . . . , ̂ Num ( r π ( ℓ -1) , r π ( ℓ ) )) . ̂ Num ( π, ˆ E ) denotes the set of approximate path counts which use the internal anchor edges ˆ E . Similarly, approximate counts ̂ Num ( π, ˆ E ) can be defined for π ∈ Π 1 ( D ) and π ∈ Π 2 ( D ) . Let Π( D ) = Π 0 ( D ) ∪ Π 1 ( D ) ∪ Π 2 ( D ) .

Let IA ( H ) denote the set of internal anchor edges in a compact subgraph H . We refer to a tuple ( H,D, Π( D ) , ⋃ π ∈ Π( D ) ̂ Num ( π, IA ( H ))) as an augmented compact subgraph with approximate counts .

Feasible solutions as a tuple of augmented compact subgraph with approximate counts. We now observe that any feasible solution involving anchoring a subset S = { ( i r , j r ) : r = 1 , . . . , s } of pairs of nodes can be viewed as a set of at most k disjoint augmented compact subgraphs with approximate counts.

Lemma 23. Any feasible solution S = { ( i r , j r ) : r = 1 , . . . , s } , with | S | ≤ k corresponds to a set of augmented compact subgraphs with approximate counts ( H 1 , D 1 , Π( D 1 ) , ⋃ π ∈ Π( D 1 ) ̂ Num ( π, IA ( H 1 ))) , . . . , ( H ℓ , D ℓ , Π( D ℓ ) , ⋃ π ∈ Π( D ℓ ) ̂ Num ( π, IA ( H ℓ )) , for some ℓ ≤ k , such that for every j , ( H j , D j ) consists of nodes with lower index than those in ( H j ′ , D j ′ ) , for j < j ′ .

Proof. (Sketch) Consider the set of all paths P which are considered, i.e., those which use the edges from S . Let v be the node with the minimum index among those in P . Consider the graph G σ ( v, k, ∅ ) . If any pair ( i s , j s ) ∈ S is an internal anchor pair for G σ ( v, k, ∅ ) , define e 1 = ( i s , j s ) (this is added to IA ( H 1 ) ); this is again repeated if there is another pair ( i s ′ , j s ′ ) which is an internal anchor pair for G σ ( v, k, e 1 ) . We stop, when we have a subgraph H 1 such that no other pair in S is an internal anchor pair for H 1 . If there is a path P ∈ P with one end point u ∈ V ( H 1 ) and other end point outside H 1 , u will be added to D 1 . Finally, we construct pairings and associated approximate counts. If a path P has a segment P ′ from r 1 ∈ D 1 to r 2 ∈ D 1 which is within H 1 , we would have ( r 1 , r 2 ) as part of a pairing, and the number of such segments will contribute to the associated counts. Thus a compact subgraph ( H 1 , D 1 , Π( D 1 ) , ⋃ π ∈ Π( D 1 ) ̂ Num ( π, IA ( H 1 ))) can be constructed corresponding to the set P .

Next, consider the path segments we get once we delete all the segments in H 1 . The next compact subgraph is constructed in a similar manner.

Let S ( G,σ,k ) denote the set of possible augmented compact subgraphs with approximate counts. From Lemma 23, it follows that any feasible solution is an element of ⋃ k ′ ≤ k S ( G,σ,k ) k ′ .

Lemma 24. The number of feasible solutions is O ( exp ( k k d k +1 log( kd k ))( k log n ) k 2 ) .

Proof. (Sketch) We first bound |S ( G,σ,k ) | . For each compact subgraph ( H,D ) , the number of pairings | Π( D ) | is at most k k . For each pairing π ∈ Π( D ) , there are O (log n ) k possible approximate

counts. Therefore, |S ( G,σ,k ) | = O ( exp ( kd k +1 log( kd k ))( k log n ) k using Lemma 22. The lemma follows since the number of feasible solutions is at most k |S ( G,σ,k ) | k .

Finding a tuple of disjoint augmented compact subgraphs with approximate counts in G . We now show that we can check in polynomial time if an augmented compact subgraph exists. Thus in time O ( |S ( G,σ,k ) | poly ( n ) , we can check which augmented compact subgraphs exist. For a given augmented compact subgraph, the total number of paths can be approximated, which allows us to find the one with the maximum number.

Lemma25. Suppose we are given a tuple M 1 , . . . , M ℓ , where M j = ( H j , D j , Π( D j ) , { ̂ Num ( π ) , π ∈ Π( D j ) } ) , for some ℓ ≤ k such that for every j , ( H j , D j ) consists of nodes with lower index than those in ( H j ′ , D j ′ ) , for j < j ′ . It is possible to find a set of nodes v 1 , . . . , v ℓ , sequence of edges F (1) , . . . , F ( ℓ ) , connector sets ˆ D 1 , . . . , ˆ D ℓ , pairings Π( ˆ D 1 ) , . . . , Π( ˆ D ℓ ) , such that for each j , H j = G σ ( v j , k, F ( j ) ) , ˆ D j ⊂ V ( G σ ( v j , k, F ( j ) )) , and ⋃ π ∈ Π( ˆ D j ) ̂ Num ( π, IA ( H j ) = { ̂ Num ( π ) , π ∈ Π( D j ) } in time O ( poly ( n ) exp ( k k d k +1 log( kd k ))( k log n ) k 2 ( kd k ) t ) if G has degeneracy d and treewidth t .

̸

Proof. (Sketch) We use c j to denote a color corresponding to M j . For each node v , it is possible to decide in time O ( kd k ) if v can be colored by c j , i.e., if there exist F ( j ) , ˆ D j such that ( G σ ( v, k, F ( j ) ) , ˆ D j , ⋃ π ∈ Π( ˆ D j ) ̂ Num ( π, IA ( H j ))) = M j . If v has color c j , it has 'conflicts' with C ( v, c j ) = V ( G σ ( v, k, F ( j ) )) , i.e., we cannot have a node u ∈ C ( v, c j ) assigned color c i . Our goal is to find nodes v 1 , . . . , v ℓ such that v j can be assigned color c j , and v j ′ ̸∈ C ( v j , c j ) for any j ′ = j . If G has treewidth t , we can determine if such a coloring can be done in time O ( poly ( n )( kd k ) t ) time. The main idea for the dynamic program is the following.

Let T = ( V , E ) denote a tree decomposition of G . Each 'node' U ∈ V is a subset of V . We construct an information set I ( U ) for each U ∈ V , which will allow us to determine possible colors for each node v ∈ U , in the following manner. I ( U ) consists of a set of possible colorings x of U . We consider only x which correspond to colorings in which each color appears at most once; x can be viewed as a coloring of the subgraph induced by all the nodes in the subtree of T rooted at U . A coloring x specifies for each node u ∈ U , and for each j , an indication of the following: (1) node u is colored by c j , (2) node u is not colored by c j , but color c j that can be assigned to it, (3) node u is not colored by c j , and, moreover, c j cannot be assigned to u . This is because there exists some other node u ′ such that u ∈ C ( u ′ , c j ) . We don't keep track of node u ′ (otherwise we would end up needing n k information), but instead specify the set F and position of u in the compact graph G σ ( u ′ , k, F ) . The information set I ( U ) consists of all possible such colorings x ; note that | I ( U ) | = O (( kd k ) t ) .

We construct the information sets of all U ∈ V in a bottom-up manner. Suppose U has children U 1 , U 2 ∈ V . We construct I ( U ) from I ( U 1 ) and I ( U 2 ) , as described below. For each x ′ ∈ I ( U 1 ) , x ′′ ∈ I ( U 2 ) , we construct a coloring x , if feasible, in the following manner. For each node u ∈ U , and for each j :

If u ∈ U 1 ∩ U 2 , x is infeasible if the indicators for u in x ′ and x ′′ are inconsistent

If u is contained only in U 1 , the indicator for u in x is kept to be the same as that in x ′ ; similarly if u is contained only in U 2

̸

If u ̸∈ U 1 ∪ U 2 , we check if N ( u ) ∩ ( U 1 ∪ U 2 ) = ∅ . In this case, we choose an indicator for u in x that is consistent with the indicators for its neighbors in x ′ and x ′′ . It is also possible that we have inconsistencies, and in that case, x will not be feasible. If a node u 1 ∈ N ( u ) ∩ U 1 is in the conflict set of some node with color c j , using the position of u 1 in the associated compact subgraph, we can determine if u would also have a conflict, and if so, at what position. Similarly, if there exists a node u 2 ∈ N ( u ) ∩ U 2 . If there simultaneously exist u 1 ∈ N ( u ) ∩ U 1 , u 2 ∈ N ( u ) ∩ U 2 , both of which are in conflict sets with respect to c j , which induce inconsistent conflicts for node u with respect to c j , we would consider x infeasible; if the conflicts induced by u 1 , u 2 are consistent, we could keep that indication for node u with respect to c j .

Once the information sets I ( U ) for all U ∈ V are known, we can find a valid coloring in a top-down manner.

Lemma26. Suppose we are given a tuple M 1 , . . . , M ℓ , where M j = ( H j , D j , Π( D j ) , { ̂ Num ( π ) , π ∈ Π( D j ) } ) , for some ℓ ≤ k such that for every j , ( H j , D j ) consists of nodes with lower index than those in ( H j ′ , D j ′ ) , for j < j ′ . It is possible to determine in O ( k k ) time if the M j 's are consistent, i.e., if it is possible to find external anchor edges that are consistent for the tuples.

Theorem 8. If G has degeneracy d and treewidth t , we can get a O (1 + 1 /polylog ( n )) factor upper bound on f ( G,S ) in time O ( poly ( n ) exp ( k k d k +1 log( kd k ))( k log n ) k 2 ( kd k ) t ) , .

F Additional Experiments

In this section, we present the complete set of experimental results for private triangle counting (Algorithm 2) on all the tested networks (12 networks given in Table 2).

ca-AstroPh

ca-GrQc

0.4

80

0.2

0.1

Private Triangle Count Rel. Error

0.0

com-Amazon

com-DBLP

email-Enron

0.4

0.3

0.2

0.1

0.0

Oregon-1_G_1

Oregon-2_G_1

0.4

0.3

0.2

0.1

0.0

0.125

0.250

0.500

1.0002.000.1250.2500.500

1.000

2.000.125

0.250

0.500

1.0002.000.1250.2500.500

1.0002.000

epsilon

Alg→Approximation

Exact

0.03

Theratio of SmoothSensitivity / #

0.02

0.01

F00'0

ca-AstroPhca-CondMat

ca-GrQc

ca-HepThcom-Amazoncom-DBLP

email-Enronloc-Brightkiteloc-Gowalla Oregon-1_G_Oregon-2_G_1

Network

ca-AstroPh

ca-GrQc

1000

100

10

1

com-Amazon

com-DBLP

email-Enron

1000

Speedup

100-

10

Oregon-1_G_1

Oregon-2_G_1

1000

100

10

1

0.1250.2500.5001.0002.00@.1250.2500.5001.0002.00@.125

0.2500.5001.0002.00@.1250.2500.500

1.0002.000

epsilon

ca-AstroPh

ca-GrQc

2.0

1.5

1.0

Error

com-Amazon

com-DBLP

email-Enron

2.0

Approx.

1.5

Sensitivity

1.0

0.5

0.0

Oregon-1_G_1

Oregon-2_G_1

2.0

1.5

1.0

0.5

0.0

0.125

0.250

0.500

1.0002.000.125

0.250

0.500

1.000

2.000.125

0.250

0.5001.000

2.000.125

0.2500.5001.0002.000

epsilon

Alg—Approx.Error -.-.Gamma.Upper

Figure F shows that when approximate smooth sensitivity is used instead of exact smooth sensitivity, the accuracy of the private triangle counts measured using the metric, Triangle Count Relative Error, are the same across different privacy budgets in all but two networks (Oregon 1 and Oregon 2) (Figure 8). In these two networks, the smooth sensitivities are relatively large in comparison with the triangle counts, making the noise fluctuate much more for the approximate estimate. For the ca-HepPH dataset, the two outputs coincide.

Figure 8) shows the ratio of smooth sensitivity to actual triangle count for all datasets. Note that this ratio is large for the two datasets, Oregon 1 and 2, but small for all the other datasets.

Figure 9 shows the speed-up achieved using Algorithm 2. As seen in the figure, it is orders of magnitude faster than the exact algorithm on all the 12 datasets. In fact, on large graphs (Amazon, DBLP, Gowalla), the speedup is as high as 1 , 000 -fold. Generally, a lower ϵ requires a smaller approximation factor ( γ ), which in turn requires a larger number of iterations to reduce the error in approximation. To summarize, the majority of tested networks have speedup factors between 10 and 100 -fold across all privacy budgets ϵ .

Figure 10 shows that all approximate smooth sensitivities computed by Algorithm 2 are close to the exact smooth sensitivities. We measure this using the metric, Sensitivity Approx. Error. Approximate smooth sensitivities are always larger than the true smooth sensitivity. It is necessary since a lower value may expose the privacy due to an inadequate noise magnitude. In general, a smaller value of ϵ (higher privacy guarantee) requires a more accurate estimation of the sensitivity. It is illustrated in Figure 10 as the approximate smooth sensitivity is close to the exact smooth sensitivity in all networks at ϵ = 0 . 125 or ϵ = 0 . 25 .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the problem of releasing information about graphs (specifically, subgraph counts) under differential privacy.  This has been a widely studied problem, due to the challenging nature of ensuring that the noise added is bounded.  Many techniques (e.g., smooth sensitivity) have been developed within the privacy community motivated by such graph problems.  A particular concern is the computational cost: many techniques have a high polynomial or exponential cost, rendering them impractical -- and more costly than their non-private counterparts.  The present paper studies how approximation can adopted, in two ways: applying approximation to the sensitivity calculations, and to the quantity of interest.  In some ways, it is perhaps not surprising that bounds can be obtained by tolerating approximation, a common technique in randomized algorithms.  Recent work has started to look at the intersection of approximation and privacy (e.g., in the case of quantile computations, https://arxiv.org/abs/2201.03380), so it is natural to apply it here.  Specific results are shown for triangles, paths and trees, as subgraphs of particular interest.\n\nSTRENGTHS:\nThese are fundamental questions about the effect of privacy -- motivated by practical concerns, but perhaps contributing more to out theoretical understanding of whether we can obtain private counterparts of exact or approximate algorithms for subgraph counting.  The paper opens up the question of using approximation in different ways in the private algorithm design, and focuses attention on the computational cost of the algorithms, whereas prior work has looked primarily at the accuracy. \n\nThe technical details (in full in the supplemental material) is detailed and challenging in places, although making use of standard probability tools (e.g., Chernoff bounds).\n\nWEAKNESSES:\nThe paper takes great pains to cite recent work by Blocki et al [4] which as far as I can tell is currently an unpublished technical report.  It summarizes the difference as studying approximate sensitivity.  Still, I think it would be useful to provide more contrast.  For instance, the empirical study compares against the original work on smooth sensitivity from 2007.  Many papers relevant to private triangle counting have come since then.  In particular, it would be instructive to also compare against the black-box approach of Blocki et al to see where it falls on these axes.\n\nQUESTIONS:\nLemma 2 seems like it should be obvious: if we have an estimate of the smooth sensitivity that is always bigger except with probability delta', then we should be able to directly use this to add noise based on the estimate, and only incur delta' additional \"failure probability\" in the (eps, delta)-differential privacy bounds.  But the proof in the supplementary material works through a longer argument adapting the original proof of smooth sensitivity.  Can you comment on whether the intuitive argument above is valid. \n\nIn Defn 5, wouldn't it be quicker to say that g_f is A_f(D) clipped to the range [ (1-alpha)f(D), (1+alpha)f(D) ]?  And similarly for s_f.\n\nTheorem 2, the dependence should be on log(1/(2 delta)) ?\n\nLIMITATIONS:\nNo limitations around societal impact. \nThe paper could discuss more the limitations of the approach and future work -- Section 7 is very brief on this topic.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on the problem of counting the number of non-induced embeddings of a subgraph while ensuring differential privacy. The objective is to develop efficient DP algorithms for subgraph counting that outperform previous approaches in terms of running time.\n\nTo accomplish this goal, the authors give a method that leverages approximate smooth sensitivity as a substitute for global sensitivity in privacy mechanisms. This approach proves effective in accurately estimating smooth sensitivity for tasks such as triangle counting and path counting.\n\nIn addition, the paper presents experimental results for triangle counting using real-world data. These experiments demonstrate that the algorithm's estimation outcomes closely align with the accurate values.\n\nSTRENGTHS:\nThe problem of counting the number of subgraphs is an important question in the graph differential privacy community. \n\nThe idea of using approximate smooth sensitivity to improve the running times of current algorithms is somehow natural and interesting.\n\nWEAKNESSES:\nThe running time of the DP algorithm for (arbitrary) subgraphs with $\\ell$ edges is $O(n^{2\\ell})$, which is still quite inefficient.\n\nQUESTIONS:\nFor the DP algorithms given in the paper, the output A(D) is an approximation of the quantity that we want to compute (e.g., the number of triangles). But the approximation ratio of A(D) is never mentioned, i.e. what is the multiplicative error and what is the additive error? Furthermore, how could you compare these algorithms to previous algorithms in terms of approximation ratios?\n\nWhat is the state-of-the-art of running times of the DP algorithms for counting subgraphs with ell edges? \n\nWhy the experiments are only for triangles? What about paths or other subgraphs?\n\nLIMITATIONS:\nSee \"Weaknesses\".", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends the study of local sensitivity and smooth sensitivity for the problem of subgraph counting. Previous work has identified local sensitivity and its derivates like smooth sensitivity as a tool to calibrate noise to the given input instead of global sensitivity. However, this comes with two challenges: it is often not easy to get a good bound on local sensitivity in a computationally efficient way, and local sensitivity itself can exploit privacy (which lead to the notion of smoothed sensitivity, a DP version of local sensitivity). In this submission, the authors study how an approximation of the smooth sensitivity and higher-order local sensitivity can be used privately to release an approximation of subgraphs counts, in particular triangles, paths and arbitrary constant-size graphs.\n\nFor triangle count, they show that one can release an approximation in $O(polylog(m+n))$ time plus the time needed to approximate the triangle count non-privately, by leveraging a fast estimation scheme for the smooth sensitivity and a diamond sampling technique from non-private algorithms. For general graphs, they extend the existing first and second order analysis of local sensitiviy for triangles and cliques to the $\\ell$th order for graphs $H$ with $\\ell$ edges. They derive that one can compute an approximation of the $\\ell$th order local sensitivity and use it to release the count of copies of $H$ in the input graph in $O(n^{2\\ell})$ time for arbitrary $G$ and in time $O(n^{\\ell+1})$ if $H$ is tree. In an experiment section, they show that for several datasets of size xxK to xxxK nodes, their DP approximation algorithm for counting triangles provides one to three order of magnitude speedup, while the approximation is significantly less than a magnitude worse compared to the baseline based on exact smooth sensitivity.\n\nOverall, this paper is a nice step into a directon that I think is promising to overcome the issue with global sensitivity on graphs. The theoretical bounds on running time are not practical, but they generalize from specific instances or classes of graphs to arbitrary graphs. This is often a first step into understanding a problem better. The experiments show that the approach can also improve the state of the art where the problem is already better understood in general, i.e., in the case of triangles.\n\nSTRENGTHS:\n* The paper provides general utility bounds for counting arbitrary subgraphs that are much more tailored to the actual input than approaches based on global sensitivity.\n* The experiments indicate that algorithms also performs better in practice.\n\nWEAKNESSES:\n* The bound for general graphs and paths is impractical, as the degree of the running time polynomial scales linearly with the size of $H$.\n\nQUESTIONS:\n-\n\nLIMITATIONS:\n-", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper provides an algorithm to approximately estimate the smooth sensitivity for counting triangles in a graph, while preserving edge-level differential privacy. Compared with the existing solution for smooth sensitivity, the proposed algorithm demonstrates efficiency improvement in terms of computation.

STRENGTHS:
1. Efficiently computing/estimating the smooth sensitivity for subgraph counting is a well-motivated problem. 

2. The idea of the proposed algorithm is intuitive and the analysis seem to be sound.

WEAKNESSES:
1. It would be better if the authors can provide a table comparing their method with existing solutions. The comparison should include time complexities, tasks (e.g., triangle counting, clique counting), sample complexities, and maybe utility guarantees. 

2. The original non-private and approximate algorithm for triangle counting (the diamond sampling [2]) and also the state-of-the-art solution(s) for approximate triangle counting without DP should be included for theoretical analysis and empirical evaluation. Including them would help the readers better understand the trade-offs between privacy and utility and between utility and efficiency.

3. Some baselines are missing in the empirical evaluation. For example, the inverse sensitivity, propose-test-release, and ladder functions.

4. The algorithms in Section 5 do not have empirical evaluations.

5. A more detailed discussion on the parallelism is needed. How much does parallelism account for the efficiency improvement of the proposed solution?

QUESTIONS:
See weaknesses.

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07