Private Graph All-Pairwise-Shortest-Path Distance Release With Improved Error Rate
Abstract
Releasing all pairwise shortest path (APSP) distances among vertices on general graphs under weight Differential Privacy (DP) is known as a challenging task that has gained increasing interest recently.
Previous work achieved DP with the maximal absolute error among all published pairwise distances bounded by ˜ O ( n ) where n is the number of nodes.
Whether the approximation error can be reduced to sublinear in n is still an interesting open problem.
In this paper, we break the linear barrier on the distance approximation error in APSP release, by proposing an algorithm that releases a constructed synthetic graph privately.
Computing all pairwise distances on the constructed graph only introduces ˜ O ( n 1 / 2 ) error in answering all pairwise shortest path distances for fixed privacy parameter.
Our method is based on a novel graph diameter (link length) augmentation via constructing “shortcuts” for the paths and the use of Laplace noise with non-zero mean.
Numerical examples are also provided.
Additionally, we also propose a DP algorithm with error rate ˜ O
( k ) , which improves the error of general graphs, when the graph has small feedback vertex set number k
= o ( n 1 / 2 ) .
Introduction
In recent years there has been a growing interest in private data analysis, from academic research to industry practice.
Typically, many industrial machine learning applications consist of two steps: 1) collecting data from users; 2) training models using the collected user data.
One key question is: throughout this process, how can we extract meaningful information from the data, without leaking sensitive data of each user?
In other words, how can we prevent an adversarial attacker from inferring any user’s data, given the public information that can be accessed?
This has been the overarching question in a line of research that studies private algorithms under various settings.
The concept of differential privacy (DP) (Blum et al., 2005; Chawla et al., 2005; Dwork, 2006) has been a popular approach for rigorously defining and resolving the problem of keeping useful information for model learning, while protecting privacy for each individual.
In the traditional setting, databases D and D ′ are collections of data records and are considered to be neighboring if they are identical except for a single record (individual information).
DP requires that the output of running a randomized algorithm on D and D ′ should have very close probability distributions.
There are many types of databases, and in this work, we will specifically focus on the differential privacy of graphs.
Weight private graphs.
In general, there are three types of private graph models, regarding the nodes, the edges and the edge weights, respectively.
The node private model requires DP for two adjacent graphs differing in one node.
In the edge private model, two neighboring graphs are defined such that they only differ by one edge, while sharing the same set of nodes.
In this paper, we focus on the weight private graph model, which was first proposed by Sealfon (2016).
In the weight private graph problem, the topology of the graph is public, which means that the adjacent graphs (databases) when considering DP have same nodes and edges, while this is not the case in other two DP models.
Specifically, in the weight private model we consider two graphs as being adjacent if the difference between the sums of their edge weights is no more than one unit.
One example application of this problem is where one tries to release the transportation volume between several places of interest with privacy constraints, where the roads are regarded as edges.
In some sense, it might be hard to keep the graph structure private in practice, since one may easily obtain the map by modern tools like Google Earth.
However, in some cases, information like traffic flows could also be sensitive and needs to be kept private.
Releasing such private information is well suited for the setting of weight private graph model.
1.1 Open Problem, and Previous Results
In this paper, we study the problem of releasing all pairwise shortest-path (APSP) distances with weight privacy.
Two standard strategies in DP are: 1) adding Laplace noise to each edge, 2) adding
Laplace noise to the output distance matrix.
As discussed in Sealfon (2016), the errors of these two strategies are roughly O ( ϵ − 1 n log n ) by adding noise to each edge and O ( ϵ − 1 n (cid:112) log(1 /δ )) using DP composition theorems, respectively, where n is the number of nodes in the graph and ( ϵ, δ ) is the DP parameter.
Here, the approximation error is measured by the largest absolute difference among all released pairwise distances and the ground truth.
As we see, both of these errors are roughly O ( n ) , i.e., linear in n when ϵ and δ are fixed.
Whether we can achieve differential privacy with error sublinear in n is still an open problem, as recognized by the online repository ( https://differentialprivacy.org/open-problem-all-pairs/ ) focusing on the research of DP, quoted “... Is this linear dependence on n inherent, or is it possible to release all-pairs distances with error sublinear in n ?”
While this linear barrier seems to hold for general graphs, we can derive improved results on several special types of graph.
Firstly, for a graph with weights bounded in [0 , M ] , Sealfon (2016) picked a subset Z ∈ V of vertices as the “ k -covering set” to approximate the original graph.
Each vertex u can map to its closest vertex z u in the covering set.
For any node pair u, v , their distance is approximated by the distance between z u and z v plus O ( kM ) additional error.
Then, the author proposed to use
O ( | Z | 2 ) pairs of distances to approximate all O ( n 2 ) pairwise distances, which leads to O ( √ nM ) approximation error for fixed privacy parameters eventually.
Secondly, for grid graph with arbitrary positive weights, Fan and Li (2022) proposed to select an intermediate vertex set and divide the shortest path between any node pair into at most three parts depending on its traverse of the set.
By connecting the nodes in the intermediate set and applying the standard Laplace mechanism, the authors constructed a DP algorithm with ˜ O ( n 3 / 4 ) approximation error.
Thirdly, for trees, Sealfon (2016) gave a recursive algorithm to release all-pairs distances with error O (log 2 .
5 n ) , which was improved to O (log 1 .
5 n log 1 .
5 h ) by Fan and Li (2022), where h is the depth of tree and can be as small as O (log n ) .
The general idea was to build a collection T of subtrees of log n levels, where the subtrees in each level are disjoint to each other such that each edge appears in at most O (log n ) trees, allowing us to only add O (log n ) units of Laplace noise to each path in T .
This leads to poly log n/ϵ approximation error on trees.
1.2 Our Results and Techniques
In this paper, we propose an algorithm that is able to surpass the linear O ( n ) error of differentially privately releasing all pairwise distances on general weighted graphs.
We make use of the improved and extended techniques from Fan and Li (2022) to general graphs to obtain ˜ O ( n 1 / 2 ) approximation error.
Moreover, when the graph exhibits some special property having small feedback vertex set number, we develop a new algorithm with further improved error bound.
Breaking the linear error barrier for general graphs.
Let G ( V , E , w ) denote an undirected graph, where V is the set of nodes with | V | = n , E is the set of all edges, and w is the set of corresponding weights.
We use w ( e ) to denote the weight of a specific edge e ∈ E .
The key challenge in this problem is that, we need to bound the error of all shortest paths instead of just one.
Since the shortest paths could overlap with each other, the dependency among the shared edges (and the noises) hinders us from applying standard concentration results.
To achieve sublinear error in DP all pairwise distance release problem, our idea, intuitively, is to find “shortcuts” that will reduce the number of edges in long shortest paths (consisting of many edges).
The algorithm proceeds as follows.
Denote d ( u, v ) as the true shortest path distance between node u and v w.r.t.
graph G ( V , E , w ) .
First, we randomly sample n 1 / 2 vertices from V to form a subset of vertices V 1 ⊂ V .
We then create an edge set E
= { ( u, v ) : u ∈ V 1 , v ∈ V 1 , u ̸ = v } and assign each edge the weight w 1 ( u, v ) =
d ( u, v ) .
Denote E
=: { ( u, v ) ∈
E : u / ∈ V 1 or v / ∈
V 1 } as the edge set between nodes that are not both in V 1 .
Next, we add noise to the edges in E 0 and E 1 , respectively.
For e ∈ E 0 , we add Laplace noise following
Lap ( µ 0 , σ 0 ) where σ 0 = O (1 /ϵ ) , µ 0 = σ log( n/γ ) ; for e ∈ E 1 , the noise is from Lap ( µ 1 , σ 1 ) with σ 1 :
=
O ( n 1 / 2 (cid:112) log(1 /δ ))
/ϵ ) and µ 1 = σ 1 log( n/γ ) .
Note that, the mean of the Laplace noise is non-zero in our approach.
We will show that this novel design guarantees that the APSP distances computed directly from the synthetic and noisy graph G ′ are lower bounded by the true distances for all node pairs (w.h.p.), and the approximation error is at most ˜ O ( n 1 / 2 ) (Theorem 3.6).
Graph with small feedback vertex set number.
A feedback vertex set (FVS) of a graph, also called a loop cutset (Freuder, 1982), is a set of vertices whose removal leaves a graph without cycles, and the feedback vertex set number of a graph is the size of a smallest feedback vertex set.
The feedback vertex set problem is NP-hard (Lewis, 1978) and the best known approximation algorithm on undirected graphs is by a factor of two (Becker and Geiger, 1996).
Beyond the results derived for general weighted graphs, we also consider the problem for graphs with small feedback vertex set.
We design a new algorithm that releases the distances privately with ˜ O ( k ) error where k is the feedback vertex set number, improving the error bound when k
= o ( n 1 / 2 ) compared with the result for general weighted graphs.
1.3 More Related Work
The topic of private graphs has attracted substantial research interests in the recent years, including edge privacy, node privacy and weight privacy (Hay et al., 2009; Rastogi et al., 2009; Gupta et al., 2010; Karwa et al., 2011; Gupta et al., 2012; Blocki et al., 2013; Kasiviswanathan et al., 2013; Bun et al., 2015; Sealfon, 2016; Ullman and Sealfon, 2019; Borgs et al., 2018; Arora and Upadhyay, 2019;
Fan and Li, 2022).
Our work focuses on the weight private problem (Sealfon, 2016) as described above.
We generalize the previous work (Fan and Li, 2022) which focused on trees and grid graphs to general weighted graphs.
We are also aware of two concurrent works (Ghazi et al., 2022; Chen et al., 2022).
Both these two papers claim ˜ O ( n 1 / 2 ) error upper bound same as our result, using a similar high-level idea of constructing “shortcuts”.
However, our approach is novel and different in that the
Laplace noise we add has non-zero mean.
As will be shown in the paper, this allows us to simply publish the constructed synthetic graph, upon which standard APSP computation leads to the desired error level.
However, in both Ghazi et al. (2022) and Chen et al. (2022), the noise is centered and one has to use some specifically designed calculation (optimization) on graph to obtain the estimated distances.
In other words, only the estimated distances, but not the graph itself, can be published.
Typically, releasing the graph (ours) is more difficult than releasing the distances only.
Background: Differential Privacy (DP) on Graph
Consider a connected graph G =
( V , E , w ) , with w the collection of the weights of E .
We use w ( e ) to denote the weight of an edge e ∈ E .
Denote n = | V | , m = | E | , with n ≤ m + 1 .
We first define the notion of neighboring graphs.
In weight private model, since the nodes and edges of the graph G are unchanged, we can simply use edge weights to represent the graph.
Definition 2.1 (Neighboring) .
Graph G =
( V , E , w ) and G ′
=
( V , E , w ′ ) are called neighboring, noted as G ∼ G ′ , if
[Equation]
The Differential Privacy (DP) introduced by Dwork (2006) is defined below adapted to our problem.
Definition 2.2 (Differential Privacy (Dwork, 2006)) .
If for any two neighboring graphs G = ( V , E , w ) and G ′
=
( V , E , w ′ ) , a randomized algorithm A , and a set of outcomes
O ⊂ Range ( A ) ,
[Equation]
we say algorithm
A is ( ϵ, δ ) -differentially private.
If δ = 0 , we say that the algorithm is ϵ -DP.
The parameter δ is usually interpreted as the probability allowed for bad cases where ϵ -DP is violated.
Intuitively, differential privacy requires that after changing the database by a little (the total weights in our case), the output should not be too different from that of the original database.
Differential privacy may be achieved through the introduction of noise to the output.
To attain general ( ϵ, δ ) -DP, we may add Gaussian noise to the output.
To achieve ϵ -DP, the noise added typically comes from the Laplace distribution.
Smaller ϵ and δ indicate stronger privacy, which, however, usually sacrifices utility.
Thus, one of the central topics in the differential privacy literature is to reduce the scale of noise added, while satisfying the privacy constraint.
In this work, we focus on the popular approach to achieve DP by adding Laplace noises.
The Laplace distribution with parameter b has density function f ( x ) = 12 b exp( −| x | /b ) .
Lemma 2.1 (Dwork (2006)) .
For a function f : G → R with G the input space of graphs, define the sensitivity
[Equation]
where G, G ′ are two neighboring graphs.
Let X be a random noise drawn from Lap (0 , △ f /ϵ ) .
The
Laplace mechanism outputs
[Equation]
which achieves ϵ -differential privacy.
One important and attractive property of DP is that, different DP algorithms can be easily combined together, also with strict DP guarantee.
Lemma 2.2 (Advanced Composition Theorem (Dwork et al., 2010)) .
For any ϵ, δ, δ ′ ≥ 0 , the adaptive composition of k times ( ϵ, δ ) -differentially private mechanisms is ( ϵ ′ , kδ + δ ′ )
-differentially private for
[Equation]
which is O ( (cid:112) k log(1 /δ ′ )
· ϵ ) when k ≤ 1 /ϵ 2 .
In particular, if ϵ ′ ∈ (0 , 1) , δ, δ ′ > 0 , the composition of k times ( ϵ, 0) -differentially private mechanism is ( ϵ ′ , δ ′ )
-differentially private for
[Equation]
Finally, we introduce a basic tool regarding the Laplace random variables which will be used in the error analysis.
Lemma 2.3.
Consider n i.i.d.
random variables Z 1 , Z 2 , ..., Z n from Lap ( b ) .
With probability at least 1 − γ , ∀ 0 < γ < 1 , all n Laplace random variables have magnitude bounded by b log( n/γ ) .
Proof.
For each Z i , P ( | Z i | ≤ t ) = 1 − exp( − tb ) .
Thus,
[Equation]
Let t = b log( n/γ ) .
With n ≥ 1 , the probability becomes (1 − γn ) n ≥ 1 − γ as claimed.
Private Synthetic Graph Release for APSP Distance
We formally define the problem of interest and some basic concepts as below.
Definition 3.1.
(Approximate Distances Release)
Given an graph G ( V , E , w ) , our task is to release all pairwise shortest path distances privately.
Let ˆ d ( · , · ) be the output (approximate) distance function.
Our object is to minimize the maximal absolute error over all pairs, namely, max {| ˆ d ( u, v ) − d ( u, v ) | : u, v ∈ V } , with ˆ d ( u, v ) , ∀ u, v ∈ V being DP to weight w .
We call | ˆ d ( u, v ) − d ( u, v ) | the additive error of node pair ( u, v ) .
Definition 3.2 (Path) .
A path ˜
P v 1 ,v L between v 1 , v L ∈ V is defined as ˜ P v 1 ,v L = { ( v i , v i +1 ) , 1 ≤ i ≤
L − 1 } , the collection of edges between connected node sequence v 1 , v 2 , ..., v k , with some L ≤ n .
A segment of path ˜
P is defined as a consecutive sub-path, { ( v i , v i + 1) , s ≤ i ≤ t − 1 } for some s, t .
Definition 3.3 (Shortest path) .
Let S u,v be the set of all paths between u, v ∈ V .
For a path P ∈ S u,v and weights w , denote d ( P, W )
= (cid:80)
e ∈ P w ( e ) .
The shortest path of u, v ∈ V w.r.t.
weights w is
P u,v = argmin P ∈ S u,v d ( P, w ) , and the shortest path distance is d ( P u,v , w ) .
Definition 3.4 (Canonical shortest path) .
For a given graph G ( V , E , w ) , let V 1 ⊆
V be a subset of V .
Let P x,y =
{ x, v 1 , v 2 , ..., v L , y } be the shortest path between x, y ∈ V .
Then a canonical shortest path P V 1 x,y is defined by either of the following:
P V 1 x,y ≡ P x,y , if P x,y contains at most one vertex in V 1 ; 2.
P V 1 x,y = { x, v 1 , ..., p, q, ..., v L , y } , if p, q are the closest nodes in P x,y to x, y respectively, and p, q ∈ V 1 .
Intuitively, the canonical shortest path finds a shortcut by directly connecting two nodes in a set V 1 .
This definition will be the key in our algorithm and construction.
One important fact is that, if we connect each pair of nodes in V 1 and assign the true pairwise distance between them as the edge weight (as in our main algorithm), then d ( P u,v , w ) ≡ d ( P V 1 u,v , w ) by definition.
3.1 Challenges and the New Algorithm
First, we revisit the prior approach to release all pairwise distances in the private weight model and the challenges.
Since neighboring (total) weights differ by at most one unit in l 1 norm, the distance between any two nodes also changes by at most one unit.
Releasing a single path can be done by computing the accurate shortest path distance between pairs of inputs and adding Laplace noise proportional to 1 /ϵ , which is a trivial task in DP.
However, releasing all distances privately is much more challenging, since the Laplace mechanism requires Lap ( n 2 /ϵ ) noise (because of the nqueries), resulting in the O ( n ) error eventually.
As introduced in Section 1.1, in prior literature, there are two ways (e.g., Sealfon (2016)) to achieve this error level, either by adding noise to each edge or to the output distances.
In this paper, we will focus on the first strategy.
The simple approach is as follows: 1) add Lap (1 /ϵ ) noise to each edge, i.e., w ′ =
w + Lap (1 /ϵ ) , to get graph G ′
=
( V , E , w ′ ) ; 2) report all pairwise distances on G ′
(note that, the shortest paths on
G ′ found might be different from the true shortest paths in G ).
By the Laplace mechanism (Dwork, 2006), all the weights in graph G become differentially private.
By the post-processing property of
DP, all the output pairwise distances are also DP.
Note that there are O ( n 2 ) pairs of vertices, so the number of edges is bounded by O ( n 2 ) .
By Lemma 2.3, with probability 1 − γ , all O ( n 2 ) Laplace random variables will have magnitude bounded by (1 /ϵ ) log( n 2 /γ ) , so the length of every path in the released synthetic graph is within n log( n/γ ) /ϵ
additive error, thus roughly O ( n ) .
Statistically, the general problem can be described informally and approximately as: given a set of ni.i.d.
Laplace random variables, we want to bound the sum of the n variables among n 2 size- n subsets simultaneously.
Since the graph topology can be arbitrary, it is not hard to find examples where getting a sublinear error seems impossible, and a straightforward additive bound exactly leads to the previous O ( n ) error as mentioned above.
In this section, we propose an algorithm that leverages the concept of canonical shortest paths (Definition 3.4), which finally leads to ˜ O
n 1 / 2 ) additive approximation error.
As summarized in Algorithm 1, for a general weighted graph G ( V , E , w ) , our algorithm proceeds as follows:
1) Sample n 1 / 2 vertices from V uniformly to form set V 1 ; 2) Create an edge set E 1 :
= { ( u, v ) : u ∈ V 1 , v ∈ V 1 , u ̸ = v } .
For each e = ( u, v ) ∈
E 1 , set w ( e ) =
d ( u, v ) as the true shortest path distance; 3) Add Lap ( µ 1 , σ 1 ) noise to each edge e ∈ E 1 , i.e., w ′ = w + Lap ( µ 1 , σ 1 ) ; 4) Add Lap ( µ 0 , σ 0 ) noise to each edge e ∈ E 0 , i.e., w ′ = w + Lap ( µ 0 , σ 0 ) where E 0 : =
E \ E 1 ; 5) Obtain and release the merged graph G ′
= ( V , E ′ = E 0 ∪ E 1 , w ′ ) .
One can simply compute all pairwise distances on G ′ as the estimation.
Input: General graph G =
( V , E , w ) , private parameter ϵ, δ, γ .
ϵ ′
=
ϵ/ 2 .
Sample ( n 1 / 2 ) vertices from V uniformly and add them to V 1 .
Create an edge set E 1 :
= { ( u, v ) : u ∈ V 1 , v ∈ V 1 , u ̸ = v } .
For each e ∈ E 1 , let APSP ( e ) be the exact shortest path distance in G between ( u, v ) where e = ( u, v ) .
for each edge e ∈ E 1 do
Let σ 1 :=
(2 √ 2 n 1 / 2 (cid:112) log(1 /δ ))
/ϵ
′ and µ 1 :=
σ 1 log( n/γ ) .
Draw X e ∼ Lap ( µ 1 , σ 1 ) .
w ′ ( e ) :=
APSP ( e ) + X e .
E 0 :
=
E \ Efor each edge e ∈ E 0 do
Let σ 0 := 1 /ϵ ′ , µ 0 :
= σ 0 log( n 2 /γ ) .
Draw X e ∼ Lap ( µ 0 , σ 0 ) .
w ′ ( e ) :
= w ( e ) + X e .
Compute all pairwise distances in graph G ′
=
( V , E ′ =
E 0 ∪ E 1 , w ′ ) .
Output : All pairwise distances of G ′ .
Algorithm 1: Private all pairwise shortest paths distance release.
Comments.
Note that in Algorithm 1, the final estimated distances are calculated by using standard
APSP distance computation on graph G ′ .
One implication of this construction is that, beyond outputting the pairwise distances with privacy, we can in fact also publish the graph G ′
( V , E ′ , w ′ )
privately and allow the users to compute the private distances using standard algorithms by themselves.
This is due to the positive mean of the Laplace noises.
If we instead add zero-mean (centered) Laplace noises as in most standard approaches, then computing the APSP distances on G ′ would not ensure the desired approximation error rate.
We will provide more discussion on the role of the shifted noises at the end of this section.
Next, we provide theoretical analysis of our proposed algorithm.
Firstly, recall that V 1 contains n 1 /uniformly sampled nodes from V .
Our analysis starts with the following fact.
Lemma 3.1 (Sampling Intersection) .
Suppose U ⊆
V is a fixed vertex set and | U | = n 1 / 2 log( n 2 /γ ) .
With probability at least 1 − γ/n 2 , we have that U ∩ V 1 ̸ = ∅ .
Proof.
Suppose we pick a random vertex v from V each time, the probability that v is not in V 1 is 1 − 1 /n 1 / 2 .
Then the probability of interest is bounded by Pr [ U ∩ V 1 = ∅ ] ≤ (1 − 1 /n 1 / 2 ) ( √ n log( n 2 /γ ))
=
γ/n 2 .
3.2 Privacy Analysis and Sublinear Error Bound
Lemma 3.2.
Algorithm 1 achieves ( ϵ, δ ) -DP.
Proof.
The privacy budget is divided into two parts: (Part 1)
The noise added to E 0 .
It is obvious that for two neighboring inputs differ in the total weights of edges in E 0 by 1 , by adding Laplace noises according Lap ( µ 0 , 1 /ϵ ′ ) , it achieve ( ϵ ′ , 0) -DP.
Note that adding a constant µ 0 independent of the edge weights to the edges does not affect weight privacy.
(Part 2)
The noise added to edges in E 1 .
Similar arguments hold.
There are at most n 1 / 2 · n 1 / 2 = n pairs in E 1 .
Applying Laplace mechanism with composition theorem (Lemma 2.2), we know that adding noise following Lap ( µ 1 , σ 1 ) with σ 1 = (cid:112) 8 n log(1 /δ ) /ϵ
= 2 √ 2 n 1 / 2 (cid:112) log(1 /δ )
/ϵ
′ suffices to achieve ( ϵ ′ , δ ) -DP.
Applying simple DP composition theorem again proves the (2 ϵ ′ , δ ) -DP.
We conclude by noticing that ϵ ′ =
ϵ/ 2 .
We have the following fact that, any “long path” in G would contain at least two nodes in V 1 with high probability.
Lemma 3.3.
For a given path in G ( V , E , w ) with number of edges larger than 2 n 1 / 2 log( n 2 /γ ) , it
Proof.
Let us divide the path into three parts, such that two of them have at least n 1 / 2 log( n 2 /γ ) edges.
Based on Lemma 3.1, we have that each part above intersects with V 1 with at least one vertex, each with probability at least 1 − γ/n 2 .
As a result, by union bound, the whole path contains at least two nodes in V 1 with probability 1 − 2 γ/n 2 .
Recall G ′ =
( V , E ′ =
E 0 ∪ E 1 , w ′ ) is the constructed synthetic graph with noisy weights.
From now on we denote w as the weight set containing the constructed shortcut weights, and define
G ′ w =
( V , E ′ , w ) as the graph with nodes V , edges E ′ and weights w (notice that it is different from G ′
=
( V , E ′ , w ′ ) ).
Consider those canonical shortest paths found by the true weights w on G ′ w ( V , E ′ , w ) .
We show that each canonical shortest path P V 1 u,v , u, v ∈ V in G ′ w has error | d ( P u,v , w ′ )
− d ( P u,v , w ) | bounded by ˜ O ( n 1 / 2 ) , where d ( P u,v , w ) is defined by Definition 3.3.
Lemma 3.4.
For any u, v ∈ V , let P V 1 u,v be the canonical shortest path (Definition 3.4) found by the true weights w .
Then, it holds that | d ( P V 1 u,v , w ′ )
− d ( P V 1 u,v , w ) | =
O ( n 1 / 2 (cid:112) log(1 /δ ) log 2 ( n/γ ) /ϵ ) with probability 1 − 4 γ for ∀ u, v ∈ V .
Proof.
Firstly, we have | d ( P V 1 u,v , w ′ )
− d ( P V 1 u,v , w ) | = | (cid:80) e ∈ P V 1 u,v ( w ′ ( e ) − w ( e )) | , and w ′ ( e ) − w ( e ) is the corresponding Laplace noise added to edge e in P V 1 u,v .
The key observation in the proof is that, each canonical shortest path P V 1 u,v can be divided into three parts ( u, ..., p ) , ( p, q ) , ( q, ..., v ) , where p, q are the closest nodes in V 1 to u, v , respectively, and the edge ( p, q ) is the constructed shortcut provided by V 1 and E 1 .
Note that, P V 1 u,v also might not contain a shortcut, but this does not matter in our analysis in the sequel.
Denote P u,v, as the shortest path between u and v .
Consider two cases:
• | P u,v | ≤ 2 n 1 / 2 log( n 2 /γ ) .
In this case, we know that the total length of ( u, ..., p ) and ( q, ..., v ) is bounded by 2 n 1 / 2 log( n 2 /γ ) .
That is, P V 1 u,v contains at most 2 n 1 / 2 log( n 2 /γ ) edges from E 0 .
By Laplace mechanism, with probability 1 − γ , every edge in E 0 leads to at most O (log( n/γ ) /ϵ ) error.
Thus, edges in E 0 contribute at most O ( n 1 / 2 log 2 ( n/γ ) /ϵ ) error.
P V 1 u,v also contains at most one shortcut edges from V 1 , which gives O ( n 1 / 2 log( n/γ ) (cid:112) log(1 /δ )
/ϵ ) error, with another probability 1 − γ .
Thus, with probability 1 − 2 γ , the error is bounded by O ( n 1 / 2 log 2 ( n/γ ) /ϵ ) .
• | P u,v | > 2 n 1 / 2 log( n 2 /γ ) .
In this case, by Lemma 3.3 and union bound, we know that with probability 1 − 2 γn 2 n 2 = 1 − 2 γ , P V 1 u,v contains a shortcut ( p, q ) , and both the lengths of ( u, ..., p ) and ( u, ..., q ) are upper bounded by n 1 / 2 log( n 2 /γ ) .
The remaining proof is similar to the arguments above.
We have that with probability at least 1 − 4 γ , | d ( P V 1 u,v , w ′ ) −
d ( P V 1 u,v , w ) | is bounded by O ( n 1 / 2 log 2 ( n/γ ) /ϵ )
In summary, we have shown that with probability at least 1 − 4 γ , | d ( P V 1 u,v , w ′ )
− d ( P V 1 u,v , w ) | is bounded by O ( n 1 / 2 log 2 ( n/γ ) /ϵ ) .
The proof is complete.
Additionally, we have the following result stating that the estimated distance is no smaller than the true distance for all pairs of vertices, with high probability.
Lemma 3.5.
Let P u,v be the shortest path between u, v ∈ V on G , and P ′ u,v the shortest path found on G ′ .
Then with probability 1 − 2 γ , d ( P ′ u,v , w ′ ) ≥ d ( P u,v , w ) for all u, v ∈ V .
Proof.
By adding each edge in E 1 with noise according to Lap ( µ 1 , σ 1 ) where σ= 2 √ 2 n 1 / 2 (cid:112) log(1 /δ ) /ϵ
′ and µ 1 = σ 1 log( n/γ ) , with high probability 1 − γ , every Lap ( µ 1 , σ 1 ) variable is greater or equal to µ 1 − σ 1 log( n/γ ) ≥ 0 with probability 1 − γ based on Lemma 2.3.
Similarly, for edges in E 0 , we add to each of them Lap ( µ 0 , σ 0 ) noise where σ 0 = 1 /ϵ ′ , µ 0 = σ 0 log( n 2 /γ ) .
Then with another high probability 1 − γ , every variable according to Lap ( µ 0 , σ 0 ) is non-negative based on Lemma 2.3.
Therefore, since with high probability the added noises on all the edges are positive, the noisy shortest path distance must be larger than the true distance, i.e., d ( P ′ u,v , w ′ ) .
This proves the claim.
n
A pp r o x .
E rr o r
R a t e 10= 0.
= 0.
=empirical scale n scale n 1/2 log 2 n    n
A pp r o x .
E rr o r
R a t e 10= 0.
= 0.
=empirical scale n scale n 1/2 log 2 n
Now, we are in the position to present the main error guarantee of this work.
Theorem 3.6.
Let G =
( V , E , w ) be general graph with n vertices.
For some ϵ, δ > 0 , running
Algorithm 1 publishes the graph G ′ with ( ϵ, δ ) -differential privacy w.r.t.
the weights w .
Let ˆ d ( u, v ) be the estimated shortest path distance between u, v ∈ V computed on G ′ .
Then, with probability 1 − 4 γ , we have | ˆ d ( u, v ) − d ( u, v ) |
≤ O ( ϵ − 1 n 1 / 2 log 2 ( n/γ ) (cid:112) log(1 /δ )) for all u, v ∈ V .
Proof.
Let P ′ u,v be the shortest path between u and v in the synthetic graph G ′
( V , E ′ , w ′ ) , then trivially, d ( P ′ u,v , w ′ )
≤
d ( P V 1 u,v , w ′ ) .
By Lemma 3.4, we know that | d ( P V 1 u,v , w ′ )
− d ( P V 1 u,v , w ) | is bounded by O ( n 1 / 2 (cid:112) log(1 /δ )
log 2 ( n/γ ) /ϵ ) with probability 1 − 4 γ , ∀ u, v ∈ V .
In this event, we have d ( P ′ u,v , w ′ ) ≥ d ( P u,v , w ) based on Lemma 3.5.
Notice that, d ( P u,v , w ) ≡ d ( P V 1 u,v , w ) by definition.
Therefore, with probability 1 − 4 γ ,
[Equation]
which verifies the claim.
Comments.
As we see, the positive mean of Laplace noises implies Lemma 3.5 which provides a key inequality in the proof of Theorem 3.6.
This result in fact allows us to only consider the error on the positive side.
On the contrary, if we use mean-zero Laplace noises, then we can no longer bound | d ( P ′ u,v , w ′ ) − d ( u, v ) | by | d ( P V 1 u,v , w ′ )
− d ( u, v ) | as in Lemma 3.4, since it is possible that d ( P ′ u,v , w ′ ) ≤
d ( P V 1 u,v , w ′ ) <
d ( u, v ) .
Thus, if we simply add zero-mean noises, directly computing the APSP distances on the constructed graph G ′ would not guarantee the desired error level.
3.3 Numerical Example
We present a numerical example on a multi-stage graph to justify the theory.
We simulate a multistage graph, where each edge weight is generated from i.i.d.
uniform distribution.
The graph contains concatenated blocks, each with 1 start node, 1 end node, and 9 nodes in the middle connected to both the start node and the end node. The end node then becomes the start node of the next block.
We consider multi-stage graphs here because in some sense it is one of the “hardest” cases for private APSP computation, since many shortest paths contain O ( n ) edges.
In Figure 1, we present the empirical largest absolute error over all pairwise distances against the size of graph n .
For the theoretical bounds, we start with the first empirical error (black) associated with n = 100 , and increase at the rate of n (dash) and n 1 / 2 log 2 n (dotted), respectively.
We validate that the empirical error grows with rate slower than n 1 / 2 log 2 n , as given by our theory, for all ϵ values.
Note that the error bound becomes tighter with large edge weights (right) than with small weights (left).
When the edge weights follow Unif (10 4 , 10 5 ) which is much larger than the noise magnitude, the shortest path P ′ u,v found on G ′ would be the same as P u,v (or P V 1 u,v ) in most cases.
Thus, the LHS and the RHS of (a) in (1) would be close, making the bound tighter.
Graph with Small Feedback Vertex Set
Input: General graph G =
( V , E , w ) , private parameter ϵ, δ , ϵ ′ =
ϵ/ 3 .
Compute the FVS S using the 2-approx.
algorithm in Becker and Geiger (1996).
// Step 1)
Let G ′ be the induced subgraph of vertex set V \ S in G .
Let k := | S | .
Compute the differentially private all pairwise shortest path distances ˆ d ( u, v ) on trees/forest G ′ using Algorithm 1 in Sealfon (2016) with privacy parameter ϵ ′ .
// Step 2)
Compute d ( u, v ) for u, v ∈ S , where d ( u, v ) is the exact shortest path distance in G . // Step 3) for each u ∈ S, v ∈ S do
Let σ 1 :=
(2 √ 2 k (cid:112) log(1 /δ ))
/ϵ
Draw X e ∼ Lap (0 , σ 1 ) .
ˆ d ( u, v ) :
=
d ( u, v ) + X e .
for each vertex u ∈ S do for each vertex v / ∈ S adjacent to u do // Step 4): To deal with the edges between S and V \ S
Let σ 0 := 1 /ϵ ′ .
Draw X e ∼ Lap (0 , σ 0 ) .
w ′ ( u, v ) :
= w ( u, v ) + X e .
ˆ d ( u, v ) :
= w ′ ( u, v ) .
for each vertex u in V \ S do for each vertex v in S do //
Step 4): To deal with the case that only the last vertex of shortest path is in S ˆ d ( u, v ) :
= min p/ ∈ S { ˆ d ( u, p ) + w ′ ( p, v ) } .
for each vertex u in V \ S do for each vertex v in S do //
Step 5): To deal with the case that shortest path passes a vertex p in S ˆ d ( u, v ) :
= min p ∈ S { ˆ d ( u, p ) + ˆ d ( p, v ) } .
for each pair u, v ∈ V \ S do // Step 6): To deal with the case that shortest path passes some vertex u in S ˆ d ( u, v ) :
= min p ∈ S { ˆ d ( u, p ) + ˆ d ( p, v ) } .
Output : All pairwise distances ˆ d ( u, v ) .
In previous section, we have investigated the problem of DP all pairwise shortest distance problem on general weighted graphs.
In this section, we consider a more special type of graph with small feedback vertex set (FVS), which is a common concept in graph analysis, e.g., (Kratsch and Schweitzer, 2010;
Papadopoulos and Tzimas, 2020).
We will design a new algorithm based on FVS computation that privately releases all pairwise distances.
We will show that this method improves the error on general graphs when the FVS number is small.
To begin with, the definition of FVS is given as below.
Definition 4.1 (Feedback Vertex Set (Becker and Geiger, 1996)) .
Let G =
( V , E ) be an undirected graph.
A set X ⊆ V is called a feedback vertex set (FVS) if G \ X is a forest, where G \ X is the induced graph by V \ X .
The feedback vertex set number is the minimal cardinality over all the possible feedback sets.
In words, a feedback vertex set (FVS) of a graph is a subset of vertices X such that after removing these nodes from the graph, the subgraph induced by V \ X contains no cycles (i.e., is a forest).
The
FVS number is the smallest size of all the FVS’s.
Next, we propose a differentially private algorithm that releases the APSP distances of graphs based on the feedback vertex set computation.
The steps of Algorithm 2 are summarized below:
1) We compute a feedback vertex set S of G by using the 2-approximation algorithm in Becker and Geiger (1996), with | S | =
k ; the induced subgraph G ′ of vertex set V \ S is a forest; We use Algorithm 1 in Sealfon (2016) to obtain the private all pairwise shortest path distances in G ′ .
For any pair ( u, v ) in G whose shortest path does not pass any vertex in S , ˆ d ( u, v ) on G ′ is already a good estimation for d ( u, v ) ; 3)
For the k 2 (recall k = | S | ) pairwise distances of the node pairs in S , we can compute the APSP distances directly and add Lap (0 , σ 1 ) to achieve DP.
We still use the ˆ d ( u, v ) to represent them; 4) For each edge ( u, v ) with u ∈ S, v ∈ V \ S , we add noise according Lap (0 , σ 0 = 1 /ϵ ′ ) to w ( u, v ) .
For a pair in { ( u, v ) | u ∈ V \ S, v ∈ S } , if the shortest path between u and v does not pass other vertex in S except v , then there exist some neighbor p of v such that d G ( u, v ) =
d G ′ ( u, p ) + w ( p, v ) based on Lemma A.1, where d G ( u, v ) ,
d G ′ ( u, v ) represent the shortest path distance between u and v in G and G ′ respectively.
Hence we use ˆ d ( u, v ) :
= min p/ ∈ S { ˆ d ( u, p ) + w ′ ( p, v ) } to estimate d ( u, v ) ; 5) For a pair in { ( u, v ) : u ∈ V \ S, v ∈ S } , if the shortest path between u and v passes some vertex p ∈ S except v , then there exist p ∈ S, p ′ / ∈ S such that d G ( u, v ) =
d G ( u, p ) + d G ( p, v ) , and the shortest path between u and p does not pass any vertex in S except p based on Lemma A.1.
We obtained ˆ d ( u, p ) in step 4).
We then use ˆ d ( u, v ) :
= min p ∈ S { ˆ d ( u, p ) + ˆ d ( p, v ) } to estimate d ( u, v ) ; 6) For pairs in { ( u, v ) ∈ ( V \ S ) 2 } , if the shortest path between u and v passes some vertex p in S , then d G ( u, v ) =
d G ( u, p )+ d G ( p, v ) .
We can use ˆ d ( u, v ) :
= min p ∈ S { ˆ d ( u, p )+ ˆ d ( p, v ) } to estimate d ( u, v ) , also ˆ d ( u, p ) , ˆ d ( p, v ) had been computed in 4), 5).
The auxiliary lemma mentioned in step 4) and 5) is given Lemma A.1 in the Appendix.
4.1 Privacy and Error Analysis
Lemma 4.1.
Algorithm 2 achieves ( ϵ, δ ) -DP.
The proof is placed in the Appendix.
The error bound of Algorithm 2 is given as below.
Theorem 4.2.
Let G =
( V , E , w ) be a general graph with n vertices.
For some ϵ, δ > 0 , running
Algorithm 2 publishes all APSP distances that is ( ϵ, δ ) -differentially private w.r.t.
the weights w .
With probability 1 − 3 γ , the additive error is bounded by O ( ϵ − 1 k log( k/γ ) (cid:112) log(1 /δ )
+ ϵ − 1 (log 2 .
5 n ) log(1 /γ )) .
Comments.
We see that when k = | S |
=
o ( n 1 / 2 ) , the error in Theorem 4.2 is smaller than the
O ( n 1 / 2 log 2 n ) error in Theorem 3.6 for the general weighted graphs.
Therefore, our result implies improved error bounds when the graph has small feedback vertex set such that k
= o ( n 1 / 2 ) .
Further, if k = o (log n ) , i.e., the graph is “almost” a forest, then the first term vanishes and the error reduces to ˜ O (log 2 .
5 n ) for trees (Sealfon, 2016).
Conclusion
In literature, to achieve ( ϵ, δ ) -differential privacy (DP) in general weight private graphs, releasing all pairwise shortest path (APSP) distances incurs maximal approximation error ˜ O ( n ) , linear in the number of vertices.
Recently, Fan and Li (2022) studied this problem on grid graphs and trees, but the linear error barrier for general weighted graphs is still an open problem (Sealfon, 2020).
In this work, we propose a differentially private algorithm to release a carefully constructed graph, and computing the APSP distance on the synthetic graph achieves ˜ O ( n 1 / 2 ) error sublinear in n , thus answering the open question.
With improved and extended techniques from Fan and Li (2022), the idea of our approach is to augment the diameter of graph by a set of shortcuts along with adding shifted
Laplace noise.
Moreover, for graphs with small feedback vertex set number k , we also propose a
DP algorithm to answer all pairwise shortest path distances
each with error ˜ O
( k ) .
This improves the result for general graphs when k , the feedback vertex set number, is small.
A Missing Proofs in Section
Lemma A.1.
For a given graph G ( V , E , w ) , let G ′ be the induced subgraph of G with vertex set V \ S .
For any pair { ( u, v ) | u ∈ V \ S, v ∈ S } : 1) if the shortest path between u and v does not pass other nodes in S , then there exist a neighbor p of v such that d G ( u, v ) =
d G ′ ( u, p ) + w ( p, v ) ; 2) if the shortest path P u,v between u and v does pass other nodes in S except v , then there exist vertices p ∈ S, p ′ / ∈ S such that d G ( u, v ) =
d G ( u, p ) + d G ( p, v ) and d G ( u, p ) =
d G ′ ( u, p ′ ) + w ( p ′ , p ) .
Proof. 1) Note that the shortest path between u and v does not pass other nodes in S except v .
Let the second last vertex from u to v be p , with p / ∈ S .
Then we have d G ( u, p ) =
d G ′ ( u, p ) , thus d G ( u, v ) =
d G ( u, p )+ w ( p, v ) =
d G ′ ( u, p )+ w ( p, v ) .
2) Let p be the first vertex in S that P u,v passes through, and denote p ′ as the vertex just before p in path P u,v .
We have that d G ( u, p ′ ) =
d G ′ ( u, p ′ ) .
Then we know that d G ( u, v ) =
d G ( u, p ) + d G ( p, v ) and d G ( u, p ) =
d G ′ ( u, p ′ ) + w ( p ′ , p ) .
A.1 Proof of Lemma 4.
Proof.
The privacy budget ϵ is divided into three parts: (Part 1) For the distances between all pairs in V \ S , our method achieves ( ϵ ′ , 0)
-DP
by the result on trees from (Sealfon, 2016).
(Part 2)
For the k 2 distances of all node pairs in S , by adding to each edge weights i.i.d.
Lap (0 , σ 1 ) noises with σ 1 = (cid:112) 8 n log(1 /δ ) /ϵ
= 2 √ 2 n 1 / 2 (cid:112) log(1 /δ )
/ϵ
′ , we can achieve ( ϵ ′ , δ ) -DP
according to Lemma 2.2.
(Part
) For
each edge ( u, v ) with u ∈ V \ S, v ∈ S ,
obviously
by Laplace mechanism w ′ ( u, v ) :
= w ( u, v ) + Lap (1 /ϵ ′ ) ,
we
achieve ( ϵ ′ , 0) -DP to release all the pairwise distances
Composing three privacy budges up and using simple composition theorem of DP, we show that
Algorithm 2 achieves ( ϵ, δ ) -DP.
A.2 Proof of Theorem 4.
Lemma A.2 (All pairwise distance on trees (Sealfon, 2016)) .
For a tree T with non-negative edge weights w and ϵ > 0 , there is an ϵ -differentially private algorithm that releases APSP distances such that with probability 1 − γ , all released distances have approximation error bounded
O (log 2 .
5 n log(1 /γ ) /ϵ ) .
Proof.
(of Theorem 4.2) For u ∈ S, v ∈ S , by adding d ( u, v ) with Lap (0 , σ 1 ) noise where σ 1 = 2 √ 2 n 1 / 2 (cid:112) log(1 /δ ) /ϵ
′ , with probability 1 − γ , | ˆ d ( u, v ) − d ( u, v ) |
= O ( k (cid:112) log(1 /δ ) log( k/γ ) ∀ u, v ∈ S .
Based on Lemma A.2, we have that with probability 1 − γ , all released distances in G ′ have approximation error O (log 2 .
5 n log(1 /γ )
/ϵ ) .
For those edges ( u, v ) with u ∈ V \ S, v ∈ S , we add to each w ( u, v ) a Laplace noise according to Lap (0 , σ 0 ) with σ 0 = 1 /ϵ ′ .
Thus, with another probability 1 − γ , | w ′ ( u, v ) − w ( u, v ) | ≤ O (log( n/γ ) /ϵ ) ∀{ u ∈ V \ S, v ∈ S } .
Union bound implies that, with probability 1 − 3 γ , the total error is bounded by O ( ϵ − 1 k log( k/γ ) (cid:112) log(1 /δ ))
+
O ( ϵ − 1 (log 2 .
5 n ) log(1 /γ )) as claimed.