Near-Optimal Correlation Clustering with Privacy
Abstract
Correlation clustering is a central problem in unsupervised learning, with applications spanning community detection, duplicate detection, automated labeling and many more.
In the correlation clustering problem one receives as input a set of nodes and for each node a list of co-clustering preferences, and the goal is to output a clustering that minimizes the disagreement with the specified nodes’ preferences.
In this paper, we introduce a simple and computationally efficient algorithm for the correlation clustering problem with provable privacy guarantees.
Our additive error is stronger than those obtained in prior work and is optimal up to polylogarithmic factors for fixed privacy parameters.
Introduction
Clustering is a central problem in unsupervised machine learning.
The goal of clustering is to partition a set of input objects so that similar objects are assigned to the same part while dissimilar objects are assigned to different parts of the partition.
Clustering has been extensively studied throughout the years and many different formulations of the problem are known.
In this paper we study the classic correlation clustering problem in the context of differential privacy.
In the correlation clustering problem [BBC04] one gets a graph whose vertices are the objects to be clustered and whose edges represent clustering preferences between the objects.
More specifically, the input of the problem is a graph with positive and negative labels (and possibly with non-negative weights) on the edges, where positive edges represent similarities between vertices and negative edges represent dissimilarities.
Then the correlation clustering objective asks to minimize the sum of (the weights of) positive edges across clusters plus the sum of (the weights) of negative edges within clusters.
Thanks to its simple and elegant formulation, the problem has received much attention and it is used to model several practical applications including finding clustering ensembles [BGU13], duplicate detection [ARS09], community mining [CSX12], disambiguation tasks [KCMNT08], automated labelling [AHK + 09, CKP08] and many more.
In this paper we focus on the most studied version of the problem where all edges have unit weight.
In this case the best known algorithm [CALN22] has an approximation guarantee of 1 . 994 , which improves over a 2 .
06 -approximation due to [CMSY15] and a 2.5-approximation due to [ACN08].
Furthermore, when the number of clusters is upper-bounded by k , a polynomial-time approximation scheme is known [GG05].
In the weighted case a O (log n ) -approximation is known [DEFI06], and improving upon this would lead to a better approximation algorithm for the notoriously difficult multicut problem.
For the maximization version of the problem, where the goal is to maximize the sum of (the weights of) the positive edges within clusters plus the sum of (the weights of) the negative edges across clusters, [CGW05, Swa04] gave a 0 .
7666 -approximation algorithm for the weighted case and a PTAS is known for the unweighted case [BBC04].
Differential Privacy (DP) is the de facto standard for user privacy [DMNS06], and it is of fundamental importance to design solutions for classic unsupervised problems in this setting.
In differential privacy, the focus is on datasets, G and G ′ , that differ on a single entry.
An algorithm A is ( ϵ, δ ) - differentially private if the probabilities of observing any set of possible outputs S of A when run on two “almost identical” inputs G and G ′ are similar: Pr [ A ( G ) ∈ S ] ≤
e
ϵ
· Pr [ A ( G ′ ) ∈ S ] + δ .
Over the last decade, there have been many works considering problems related to private graphs, e.g., [HLMJ09, RHMS09, GLM + 10, GRU12, BBDS13, KRSY11, KNRS13, BNSV15, AU19,
US19, BCSZ18, EKKL20, BEK21, NV21, FHS21].
We briefly review two possible definitions of privacy in graphs.
Edge Privacy.
In the edge privacy setting, two datasets are considered to be close if they differ on a single edge.
[HLMJ09] introduced a differentially edge-private algorithm for releasing the degree distribution of a graph.
They also proposed the notion of differential node privacy and highlighted some of the difficulties in achieving it.
[GRU12] showed how to answer cut queries in a private edge model.
[BBDS12] improved the error for small cuts.
[GLM + 10] showed how to privately release a cut close to the optimal error size.
[AU19] studied the private sparsification of graphs, which was exemplified by a proposed graph meta-algorithm for privately answering cut-queries with improved accuracy.
[EKKL20] studied the problem of private synthetic graph release while preserving all cuts.
Recently, [NV21, FHS21] proposed frameworks for the private densest subgraph problem.
Node Privacy.
Node differential privacy requires the algorithm to hide the presence or absence of a single node and the (arbitrary) set of edges incident to that node.
However, node-DP is often difficult to achieve without compromising accuracy, because even very simple graph statistics can be highly sensitive to adding or removing a single node [KNRS13, BBDS13, BCSZ18].
Differentially private clustering has been extensively studied [BDL + 17, CNX20, GKM20, CGKM21,
LS20].
Nevertheless, until very recently no theoretical results were known for differentially private correlation clustering.
In this context, the privacy is on the edges of the graph: namely, two graphs on the same set of vertices are adjacent if they differ by exactly one edge.
In a recent work, [BEK21] obtained the first differentially private correlation clustering algorithm with approximation guarantees using differentially private synthetic graph release [GRU12, EKKL20].
The framework proposed by [BEK21] is very elegant and general as it allows one to run any nonprivate correlation clustering approximation algorithm on a modified version of the input graph that ensures privacy.
Namely, any α -approximation algorithm to correlation clustering leads to a differentially private approximation algorithm with multiplicative approximation α and additive approximation O ( n 1 . 75 /ϵ ) .
This applies to the more general weighted version of the problem.
In the same paper, the authors also obtain an Ω( n/ϵ ) lower bound on the additive approximation of differentially private algorithms, even on unweighted graphs that consist of a single path.
However, the framework from [BEK21] is rather impractical, and the additive error is far from matching the lower bound on the additive approximation.
These results prompt the natural question of determining the best approximation guarantees (multiplicative and additive) that are possible under differential privacy.
As observed by [BEK21], instantiating the exponential mechanism [MT07] over the space of all clusterings yields an algorithm with additive error O ( n log n ) .
However, it is not known how to efficiently sample from the exponential mechanism for correlation clustering.
[BEK21] state finding a polynomial-time algorithm that matches the additive error of the exponential mechanism as an "exciting open problem given the prominent position correlation clustering occupies both in theory and practice".
Our Results and Technical Overview.
In this paper we present a new differentially private algorithm for the unweighted setting which achieves a constant multiplicative approximation and a nearly optimal O ( n log 2 n ) additive approximation (for fixed privacy parameters).
More precisely, we show:
Theorem 1.1.
For any ϵ and δ there is an algorithm for min-disagree correlation clustering on unweighted complete graphs that is ( ϵ, δ ) -differentially private and returns a solution of cost O (1) ·
OPT + O (cid:16) n log 2 n log(1 /δ ) ϵ(cid:17) .
Our algorithm is given in Section 3 as Algorithm 1.
Its privacy is proved in Section 4 (Theorem 4.5), and Section 5 is devoted to the approximation guarantees (Theorem 5.5).
The lower bound on the additive approximation in [BEK21] does not preclude an ( ϵ, δ ) -DP algorithm for correlation clustering on complete graphs with sublinear error.
More precisely, [BEK21] show an Ω( n/ϵ ) lower bound for weighted paths and an Ω( n ) lower bound for unweighted paths – both non-complete graphs – against pure ( ϵ, 0) -DP algorithms.
Nevertheless, we prove that linear error is necessary even for complete unweighted graphs and ( ϵ, δ ) -privacy, showing that our algorithm is indeed near-optimal.
Theorem 1.2.
Any ( ϵ, δ ) -differentially private algorithm for correlation clustering on unweighted complete graphs has additive error Ω( n ) , assuming ϵ ≤ 1 and δ ≤ 0 .
The proof of Theorem 1.2 is given in Appendix C.
Our approach follows the recent result of [CALM + 21] for obtaining the first O (1) -rounds, O (1) - approximation algorithm for correlation clustering in the parallel setting.
However, to obtain our bounds in the context of differential privacy we introduce several critical new ideas.
At a high level, the algorithm of [CALM + 21] trims the input graph in two steps.
First, it only keeps the positive edges between those vertices that have very similar neighborhoods of positive edges (such pairs of vertices are said to be in agreement ).
More precisely, for two nodes to be in agreement the size of the intersection of the positive neighborhoods should be larger than some threshold β times the positive degrees of each endpoint.
Second, it removes the positive edges whose endpoints have lost a significant fraction of its positive neighbors during the first step (such a vertex is called a light vertex).
Finally, the resulting clusters are given by the connected components induced by the remaining positive edges.
Our approach consists in making each of the above steps (agreement computation, light vertex computation, connected components) differentially private.
A natural way to make the agreement computation differentially private is to add Laplace noise to the size of the intersection of the neighborhoods for each pair of vertices u, v and to decide that u, v are in agreement if the noisy intersection size is larger than β times the positive degrees of u and v .
One of the crucial challenges here is to make sure that the total amount of noise needed to make the entire procedure differentially private is bounded, so that we can still obtain strong approximation guarantees.
The second step, the computation of light vertices, can be made differentially private in a very natural way: simply add Laplace noise to the degree of each vertex after the removal of the edges whose endpoints are not in agreement and decide whether a vertex is light based on the noisy degree.
The third step, the connected components computation, is the most challenging.
Here we need to argue that computing connected components of the graph induced by the positive edges is differentially private.
In other words, we have to show that the graph induced by these edges has no “bridge”; that is, there is not a single positive edge whose removal would increase the number of connected components.
This is done by establishing new properties of the algorithm and showing that if all the previous steps have succeeded, the presence of a bridge is in fact a very unlikely event.
Moreover, to guarantee privacy, we must carefully modify the way we treat light vertices, as well as those of low degree.
Discussion of Recent Work.
We note that the work of [CALM + 21] has been followed by the work of [AW22], who improved it in the context of streaming algorithms; however, it is not clear that there would be a benefit in using the framework of [AW22], whether in terms of running time, multiplicative, or additive approximation.
In concurrent and independent work, Liu [Liu22] proposed an ( ϵ, δ ) -DP algorithm that achieves a multiplicative approximation of O (log n ) and an additive error of ˜ O ( n 1 . 5 log 2 ( n/δ ) √ ϵ ) for general weighted graphs (assuming constant ϵ and δ ), improving
This result arises via a more careful analysis of differentially private synthetic graph release in the case of correlation clustering.
Furthermore, for unweighted complete graphs Liu obtained an algorithm with constant multiplicative approximation and an additive error of O ( n log 4 n · √ ∆ ∗ + log n ) for fixed privacy parameters, where ∆ ∗ is the maximum degree of positive edges in the graph.
The latter is a pivot-based algorithm augmented with Laplace noise.
This results in an O ( n 1 .
5 log 4 n ) worst-case additive error.
Our algorithm yields an additive error of O ( n log 2 n ) (Theorem 1.1), which significantly improves upon the result of Liu.
Preliminaries
Correlation clustering.
In the min-disagree variant of the correlation clustering problem in complete graphs one receives as input a complete signed graph G =
( V, E + , E − ) , where E + (resp., E − ) denotes the set of “+” edges (resp., “-”), and the objective is to compute a clustering C
= { C 1 , . . .
, C t } of V of minimum cost, which is the number of “-" edges whose endpoints are part of the same cluster plus the number of “+" edges whose endpoints belong to the distinct clusters.
In the sequel we will use G =
( V, E ) to refer to G =
( V, E + , E − ) (with E = E
+ ).
Differential privacy.
The definition of differential privacy (DP) is due to [DMNS06]; we use the precise formulation introduced by [Dwo06].
Definition 2.1 (Differential Privacy) .
A mechanism (randomized algorithm)
M with domain G and range M , which we will write as M : G → M , is ( ϵ, δ ) -differentially private if for any two adjacent datasets G, G ′ ∈ G and set of outcomes S ⊆
M we have
[Equation]
Recall that in this work, two datasets (graphs) are adjacent if they have the same set of vertices and differ by one edge.
An important property of differential privacy is that we can compose multiple differentially private subroutines into a larger DP algorithm with privacy guarantees.
Lemma 2.2 ([DR + 14], Theorem B.1) .
Let M 1 : G → M 1 be a randomized algorithm that is ( ϵ 1 , δ 1 ) -DP.
Further let M 2 : G × M 1 → M 2 be a randomized algorithm such that for every fixed m 1 ∈ M 1 , the mechanism G ∋ G (cid:55)→ M 2 ( G, m
) ∈
M 2 is ( ϵ 2 , δ 2 ) -DP.
Then the composed mechanism G ∋ G (cid:55)→ M 2 ( G, M 1 ( G ))
∈
M 2 is ( ϵ 1 + ϵ 2 , δ 1 + δ 2 ) -DP.
We also use the following property to analyse the privacy of our algorithm.
Its proof can be found in
Appendix A.1.
Lemma 2.3.
Let M 1 : G → M 1 be a randomized algorithm that is ( ϵ, δ ) -DP.
Suppose B ⊆
Mis a set of "bad outcomes" with Pr [ M 1 ( G ) ∈ B ] ≤ δ ∗ for any G ∈ G .
Further let M 2 : G ×
M 1 → M 2 be a deterministic algorithm such that for every fixed "non-bad" m 1 ∈ M 1 \ B we have M 2 ( G, m 1 ) =
M 2 ( G ′ , m 1 ) for adjacent G, G ′ ∈ G .
Then the composed mechanism G ∋ G (cid:55)→ M 2 ( G, M 1 ( G ))
∈
M 2 is ( ϵ, δ + δ ∗ ) -DP.
If we have k mechanisms that are ( ϵ, δ ) -DP, their (adaptive) composition is ( kϵ, kδ ) -DP.
However, it is also possible to reduce the linear dependency on k in the first parameter to roughly √ k by accepting higher additive error.
Theorem 2.4 (Advanced Composition Theorem [DRV10]) .
For all ϵ, δ ′ ≥ 0 , an adaptive composition of k ( ϵ, 0) -differentially private mechanisms satisfies ( ϵ ′ , δ ′ )
-differential privacy for
[Equation]
Let Lap( b ) be the Laplace distribution with parameter b and mean 0 .
We will use the following two properties of Laplacian noise.
Fact 2.5.
Let Y ∼ Lap( b ) and z > 0 .
Then
[Equation]
and
[Equation]
Theorem 2.6 ([DR + 14], Theorem 3.6) .
Let f : G
→
R k be a function.
Denote by ∆ f its ℓ 1sensitivity , which is the maximum value of ∥ f ( G ) − f ( G ′ )
∥ 1 over adjacent datasets G, G ′ ∈ G .
Then f + ( Y 1 , ..., Y k ) , where the variables Y
i ∼ Lap(∆ f/ϵ ) are iid, is ( ϵ, 0) -DP.
Graph notation.
Given an input graph G and a vertex v , we denote its set of neighbors by N ( v ) and its degree by d ( v ) =
| N ( v ) | .
As in [CALM + 21], we adopt the convention that v ∈ N ( v ) for every v ∈ V (one can think that we have a self-loop at every vertex that is not removed at any step of the algorithm).
Private Algorithm for Correlation Clustering
In this section we formally define our algorithm, whose pseudo code is available in Algorithmtogether with Definition 3.1.
The algorithm uses a number of constants, which we list here for easier reading and provide feasible settings for their values:
• ϵ > 0 and δ ∈ (0 , 12 ) are user-provided privacy parameters.
• β and λ , used the same way as in [CALM + 21], parametrize the notions of agreement (Definition 3.1) and lightness (Line 3), respectively.
For the privacy analysis, any β, λ ≤ 0 . 05 would be feasible.
These parameters also control the approximation ratio, which is O (1 / ( βλ )) assuming that β , λ are small enough; as in [CALM + 21], one can set e
g. β = λ
= 0 .
8 / 36 ≈ 0 .
• ϵ agr , δ agr and γ are auxiliary parameters that control the noise used in computing agreements; they are functions of ϵ and δ and are defined in Definition 3.1.
• β ′ and λ ′ are used in the privacy analysis; both can be set to 0 .
• T 0 is a degree threshold; we return vertices whose (noised) degree is below that threshold as singletons, which incurs an additive loss of O ( T 0 n ) in the approximation guarantee.
We set
[Equation]
where for the privacy analysis we require T 1 to be a large enough constant; namely, one can take the maximum of the right-hand sides of (6), (7), (8), (9), (10), (11), (14), and (15); asymptotically in terms of ϵ and δ , this is of the order O (cid:0) ln(1 / ( ϵδ ))
2 ln(1 /δ )
ϵ − 2 (cid:1) (assuming ϵ ≤ O (1) ).
To additionally obtain a constant-factor approximation guarantee, we further require a polylogarithmic T 1 , namely of the order O (log 2 n log(1 /δ )
ϵ − 2 ) – see
Eq.
(17) in the proof of Lemma 5.3.
The following notion is central to our algorithm and is used as part of Line 2 of Algorithm 1.
Definition 3.1 (Noised Agreement) .
Let us define ϵ agr = ϵ/ 5 . 8 , δ agr = δ/ 8 , and
[Equation]
(Note that γ ≥ √ 2 .)
Further let H be as defined in Line 1 of Algorithm 1.
For each pair of vertices u, v ∈ H , let E u,v be an independent random variable such that
[Equation]
We say that two vertices u ̸
= v ∈ H are in i -noised agreement if | N ( u ) △ N ( v ) | + E u,v < iβ · max( d ( u ) , d ( v )) .
If u and v are in 1 -noised agreement, we also say that u and v are in noised agreement; otherwise they are not in noised agreement.
We say that an edge ( u, v ) is in noised agreement if its endpoints are in noised agreement.
Note that we do not require that ( u, v ) ∈ E , although Algorithm 1 will only look at agreement of edges.
Analysis of Privacy
Our analysis proceeds by fixing two adjacent datasets G , G ′
(i.e., G and G ′ are graphs on the same vertex set that differ by one edge) and analyzing the privacy loss of each step of the algorithm.
We
Input : G =
( V, E ) : a graph ϵ , δ : privacy parameters ϵ
H
∈ V : ˆ d ( v ) ≥ compose steps up to Line 3 by repeatedly invoking Lemma 2.2, which allows us to assume, when analyzing the privacy of a step, that the intermediate outputs (states / randomness of the algorithm) up to that point are the same between the two executions.
Finally, we use Lemma 2.3 to argue that if the intermediate outputs before Line 4 are the same, then the output of Line 4 (i.e., the final output of
Algorithm 1) does not depend on whether the input was G or G ′ , except on a small fragment of the probability space that we can charge to the additive error δ .
We begin the analysis by reasoning about Line 1.
Lemma 4.1.
Consider Line 1 as a randomized algorithm that outputs H .
It is ( ϵ/ 4 , 0) -DP.
Proof.
The sensitivity ∆ d of the function d is 2 , as adding an edge changes the degree of two vertices by 1 .
Therefore, by Theorem 2.6, ˆ d is ( ϵ/ 4 , 0) -DP.
Furthermore, H is a function that only depends on the input (set of edges) deterministically via ˆ d .
Now we fix adjacent G , G ′ ; in this proof we will think that the domain of Algorithm 1 is G = { G, G ′ } and the notion of “ ( · , · ) -DP” always refers to these two fixed inputs.
Denote { ( x, y ) } =
E ( G ) △ E ( G ′ ) to be the edge on which G and G ′ differ.
Lemma 4.2.
Under fixed H , consider Line 2 as a randomized algorithm that, given G or G ′ , outputs the noised-agreement status of all edges in E ( G ) ∪ E ( G ′ ) .
It is (2 . 9 ϵ agr , 2 δ agr ) -DP.
Note that Algorithm 1 only computes the noised-agreement status of edges that are present in its input graph (which might be the smaller of the two), but without loss of generality we can think that it computes the status of all edges in E ( G ) ∪ E ( G ′ )
(and possibly does not use this information for the extra edge).
In the proof of Lemma 4.2 we apply the Advanced Composition Theorem to the noised-agreement status of edges incident on x or y to argue that the Laplacian noise E u,v of magnitude roughly (cid:112) max( d ( u ) , d ( v )) is sufficient.
Proof.
We will show that the sequence
[Equation]
(which determines the noised-agreement status of these edges) has the desired privacy guarantee.
Define E x to be those edges in E ( G ) ∪ E ( G ′ ) that are adjacent to x , but are not ( x, y ) , and E y similarly.
The sequence (2) can be decomposed into four parts (with independent randomness): on ( E ( G ) ∪ E ( G ′ ))
\ ( E x ∪ E y ) , on E x , on E y , and on ( x, y ) .
For the first part , the function
[Equation]
has sensitivity 0 , as for these edges we have { u, v } ∩ { x, y } = ∅ .
For the second part , we will show that the sequence
[Equation]
is (1 . 2 ϵ agr , 1 .
δ agr )
-DP.
To that end, we use the Advanced Composition Theorem (Theorem 2.4).
Let k = | E x | ; we have k ≤ d ( x ) (note that d is the degree function of the input graph, which might be the smaller of G , G ′ , but we also have ( x, y )
̸∈ E x ).
Thus the sequence (3) can be seen as a composition of k functions (each with independent randomness), each of which is a sum of a function | N ( x ) △ N ( v ) |
[Equation]
β · max( d ( x ) , d ( v )) , which has sensitivity at most 1 + β , and Laplace noise E x,v , which has magnitude at least
[Equation]
(where we used max( d ( x ) , d ( v )) ≥ d ( x ) ≥ k ).
Define ϵ x to be the inverse of this value, i.e.,
[Equation]
Thus by Theorem 2.6 each of the k functions is ((1 + β )
ϵ x , 0) -DP.
By Theorem 2.4, sequence (3) is ((1 + β ) ϵ ′ , δ agr ) -DP, where
[Equation]
, where for the first inequality we used that e ϵ x (1+ β ) − 1 ≤ 2 ϵ x for ϵ x ∈ [0 , 1] since β ≤ 0 . 05 , and the last equality follows by substituting the value of γ (see Definition 3.1) and reducing; our setting of γ is in fact obtained by solving the quadratic equation
[Equation]
Finally, we have 1 + β ≤ 1 . 2 , and thus the sequence (3) is (1 . 2 ϵ agr , δ agr ) -DP.
The third part is analogous to the second.
For the fourth part , the sensitivity of the function | N ( x ) △ N ( y ) | − β · max( d ( x ) , d ( y )) is 2 + β (when edge ( x, y ) is added, x and y disappear from N ( x ) △ N ( y ) , and max( d ( x ) , d ( y )) increases by 1 ).
The Laplace noise E x,y yields ( ϵ ∗ , 0)
-differential privacy with
[Equation]
where we used that β ≤ 0 .
2 , γ ≥ √ 2 and δ agr ≤ 0 .
58 < 0 .
Finally, we have a composition of four mechanisms with respective guarantees (0 , 0) , (1 . 2 ϵ agr , δ agr ) (twice), and (0 .
5 ϵ agr , 0) ,
and we can conclude the proof of the lemma using Lemma 2.2.
Next we turn our attention to Line 3.
Lemma 4.3.
Under fixed noised-agreement status of all edges in E ( G ) ∪ E ( G ′ ) , consider Lineas a randomized algorithm that, given G or G ′ , outputs the heavy/light status of all vertices.
It is ( ϵ/ 4 , 0) -DP.
Proof.
Under fixed noised-agreement status of all edges in E ( G ) ∪ E ( G ′ ) , the function ( l ( v ) − λ · d ( v ) : v ∈ V ) is deterministic and we can reason about its sensitivity, which is at most 2 max( λ, 1 − λ )
≤(when edge ( x, y ) is added, the degrees d ( x ) and d ( y ) increase by 1 , and l ( x ) , l ( y ) possibly increase by 1 if x and y are not in noised agreement).
Therefore the sequence (ˆ l ( v ) − λ · d ( v ) : v ∈ V ) is ( ϵ/ 4 , 0) -DP by Theorem 2.6, and it determines the heavy/light status of all vertices.
Now we analyze the last line (Line 4).
Theorem 4.4.
By a state let us denote the noised-agreement status of all edges in E ( G ) ∪ E ( G ′ ) and heavy/light status of all vertices.
Under a fixed state, consider Line 4 as a deterministic algorithm that, given G or G ′ , outputs the final clustering.
Then this clustering does not depend on whether the input graph is G or G ′ , except on a set of states that arises with probability at most 34 δ (when steps before Line 4 are executed on either of G or G ′ ).
Appendix A.2 is devoted to the proof of Theorem 4.4.
Here, to get a flavor of the arguments, let us showcase what happens in the case when x and y are both heavy.
If they are not in noised agreement, we are done; otherwise, the edge ( x, y ) impacts the final solution only if x and y are not otherwise connected in the sparsified graph ˜ G .
However, they are in fact likely to have many common neighbors in ˜ G , as they are in noised agreement and both heavy.
Indeed, if all noise were zero, this would mean that N ( x ) ∩ N ( y ) is a large fraction of max( d ( x ) , d ( y )) and that x and y do not lose many neighbors in Line 2.
We show that with probability 1 − O ( δ ) , all relevant noise is below a small fraction of max( d ( x ) , d ( y )) , and the same argument still applies.
To get this, it is crucial that the agreement noise E u,v is of magnitude only roughly (cid:112) max( d ( u ) , d ( v ))
(as opposed to e.g. max( d ( u ) , d ( v )) ).
Once we have Theorem 4.4, we can conclude:
Theorem 4.5.
Algorithm 1 is ( ϵ, δ ) -DP.
Proof.
We repeatedly invoke Lemma 2.2 to argue that the part of Algorithm 1 consisting of Lines 1– (that outputs noised-agreement and heavy/light statuses) is ( ϵ/ 4 , 0)
+ (2 . 9 ϵ agr , 2 .
δ agr )
+ ( ϵ/ 4 , 0) =
( ϵ, δ/ 4)
-DP by Lemmas 4.1 to 4.3 (recall the setting of ϵ agr and δ agr in Definition 3.1).
To conclude the proof, we argue about the last step using Lemma 2.3 and Theorem 4.4, which incurs a privacy loss of (0 , 34 δ ) .
Analysis of Approximation
For vectors β ∈ R V × V ≥and λ ∈ R V ≥ 0 , let A LG -CC ( β, λ ) be the algorithm from [CALM + 21] that uses β u,v to decide an agreement between u and v and uses λ v to decide whether v is light or heavy.
Let A LG -CC ( β, λ, E rem ) (stated as Algorithm 2) be a variant of A LG -CC ( β, λ ) that at the very first step removes E rem , then executes the remaining steps, and finally (as in Algorithm 1) outputs light vertices as singleton clusters.
Algorithm 2: A LG -CC ( β, λ, E rem ) , used for the approximation analysis.
Input : G =
( V, E ) : a graph β ∈ R V × V ≥: agreement parameter λ ∈ R V ≥ 0 : threshold for light vertices
E rem : a subset of edges to be removed
The strategy of our proof is to map the behavior of Algorithm 1 to the that of A LG -CC ( β, λ, E rem ) for carefully chosen β, λ , and E rem .
We remark that A LG -CC is not actually executed by our DPapproach, but it is rather a hypothetical algorithm which, when appropriately instantiated, resembles approximation guarantees of Algorithm 1.
Moreover, A LG -CC has similar structure to the approach of [CALM + 21], enabling us to reuse some of the results from that prior work to establish approximation guarantees for A LG -CC.
We emphasize that the approximation guarantee from prior work we build on is invoked in a black-box manner and the analysis presented in this paper is self-contained.
We begin by showing certain monotonicity properties with respect to input parameters β and λ of
A LG -CC.
Given two vectors x and y labeled by a set S , we say that x ≤ y iff x s ≤ y s for each s ∈ S .
Lemma 5.1.
Let β L , β U ∈ R V
×
V
≥and
λ
L , λ U ∈ R V ≥ 0 such that β U ≥ β L and λ U ≥ λ L .
(i)
If u and v are in the same cluster of A LG -CC ( β L , λ L , E rem ) , then u and v are in the same cluster of A LG -CC ( β U , λ U , E rem ) .
(ii)
If u and v are in different clusters of A LG -CC ( β U , λ U , E rem ) , then u and v are different clusters of A LG -CC ( β L , λ L , E rem ) .
The proof of Lemma 5.1 is given in Appendix B.1.
We now derive the following claim that enables us to sandwich cost( A LG -CC ( β, λ, E rem )) between two other instances of A LG -CC.
Lemma 5.2.
Let β L , β, β U ∈ R V
× V ≥and λ L , λ, λ U ∈ R V ≥
such that β U ≥ β ≥ β L and λ U ≥ λ ≥ λ L .
Then cost( A LG -CC ( β, λ, E rem )) ≤ cost( A LG -CC ( β U , λ U , E rem ))
+ cost( A LG -CC ( β L , λ L , E rem )) .
Proof.
We first upper-bound the cost of A LG -CC ( β, λ, E rem ) incurred by “-” edges.
If a “-” edge { u, v } adds to the cost of clustering, it is because u and v are in the same cluster.
By Lemma 5.1 ((i)), if u and v are in the same cluster of A LG -CC ( β, λ, E rem ) , then they are in the same cluster of
A LG -CC ( β U , λ U , E rem ) as well.
Hence, the cost of A LG -CC ( β, λ, E rem ) incurred by “-” edges is upper-bounded by cost( A LG -CC ( β U , λ U , E rem )) .
In a similar way, we upper-bound the cost of A LG -CC ( β, λ, E rem ) incurred by “+” edges by cost( A LG -CC ( β L , λ L , E rem )) .
If a “+” edge { u, v } adds to the cost of clustering, it is because u and v are in different clusters.
By Lemma 5.1 ((ii)), if u and v are in different cluster of A LG -CC ( β, λ, E rem ) , then they are in different clusters of A LG -CC ( β L , λ L , E rem ) as well.
Hence, the cost of the output of A LG -CC ( β, λ, E rem ) incurred by “+” edges is upper-bounded by cost( A LG -CC ( β L , λ L , E rem )) .
We now analyze the effect of removing edges incident to vertices which are not in H defined on
Line 1 of Algorithm 1.
To simplify the analysis, we first ignore the step that outputs light vertices as singletons (Line 4 of Algorithm 1 and Line 4 of Algorithm 2).
For a threshold T ∈ R ≥ 0 , let E ≤ T a subset of edges incident to vertices of degree at most T .
Lemma 5.3.
Let Algorithm 1’ be a version of Algorithm 1 that does not make singletons of light vertices on Line 4.
Assume that 5 β +2 λ < 1 / 1 . 1 and also assume that β and λ are positive constants.
With probability at least 1 − n − 2 , Algorithm 1’ provides a solution which has O (1) multiplicative and O (cid:16) n · (cid:16) log n ϵ
[Equation]
The proof of Lemma 5.3 is given in Appendix B.2.
Lemma 5.3 does not take into account the cost incurred by making singleton-clusters from the light vertices, as performed on Line 4 of Algorithm 1.
The next claim, whose proof is deferred to Appendix B.3, upper-bounds that cost as well.
Lemma 5.4.
Consider all lights vertices defined in Line 4 of Algorithm 1.
Assume that 5 β + 2 λ < 1 / 1 . 1 .
Then, with probability at least 1 − n − 2 , making as singleton clusters any subset of those light vertices increases the cost of clustering by O (OPT / ( β · λ ) 2 ) , where OPT denotes the cost of the optimum clustering for the input graph.
Combining Lemmas 5.3 and 5.4, we derive our final approximation guarantee.
Theorem 5.5.
Assume that 5 β + 2 λ < 1 / 1 . 1 and also assume that β and λ are positive constants.
Then, with probability at least 1 − n −cost
[Equation]
Future Work
It is natural to ask whether our algorithm is scalable and whether it might achieve good approximation for real-world datasets.
To begin with, one might worry that our theoretical upper bound on the multiplicative approximation ratio is a large constant.
However, the practical approximation performance of the algorithm [CALM + 21], which we build upon, has been shown to be very good (superior to pivot-based algorithms), even in regimes of parameters β
and λ where the theoretical bounds are not guaranteed to hold.
Moreover, our algorithm is efficient and easy to implement.
Unfortunately, the additive approximation term is problematic.
Recall that we return as singletons all vertices whose (noised) degree is below a threshold T 0 .
To get our theoretical bounds on approximation, we require T 0 to be polylogarithmic in n .
Motivated by the experiments of [CALM + 21], we could forgo these bounds in lieu of hoping for good practical performance.
However, we cannot proceed similarly with respect to privacy; and T 0 still needs to be a large constant in order to obtain our privacy guarantees (see Section 3 for more details on T 0 ).
Due to this, for most settings of ϵ and δ , unless the dataset is very dense, the output of our algorithm will be close to that of the trivial (0 , 0)
-DP algorithm that returns only singleton clusters.
We remark that the other known works on differentially private correlation clustering also do not provide experimental evaluations, and they seem difficult to use in practice.
The algorithm [BEK21] involves approximately solving an exponential-size linear program using semidefinite programming inside a separation oracle.
Liu [Liu22], similarly to us, outputs as singletons all vertices with degree below a large polylogarithmic threshold.
We believe that designing a practically efficient and scalable algorithm for differentially private correlation clustering is a compelling open research direction.
On the theoretical side, it would be interesting to see whether our additive approximation bound can be improved in terms of the log 2 n factor, as well as the quadratic dependence on 1 /ϵ .
Also, perhaps our lower bound (Theorem 1.2) can be strengthened, e.g. to Ω( n/ϵ ) as in [BEK21].
Furthermore, as [CALM + 21] can be implemented in O (1) parallel (MPC) rounds, it is natural to ask whether this approach can lead to an algorithm that is both parallel and private.
Finally, we do not address the general weighted setting (nor the maximum-agreement version) of correlation clustering in this paper; in particular, finding a DP algorithm with optimal additive guarantee for weighted inputs is an interesting follow-up question.