Title: Streaming Algorithms and Lower Bounds for Estimating Correlation Clustering Cost
Venue: NeurIPS 2023
Accepted: yes
Keywords: Correlation Clustering

Graph Streaming Algorithms

Large-scale Clustering

Graph Learning
Abstract: Correlation clustering is a fundamental optimization problem at the intersection of machine learning and theoretical computer science. 
Motivated by applications to big data processing, recent years have witnessed a flurry of results on this problem in the streaming model. 
In this model, the algorithm needs to process the input $n$-vertex graph by making one or few passes over the stream of its edges and using a limited memory, much smaller than the input size. 

All previous work on streaming correlation clustering have focused on semi-streaming algorithms with $\Omega(n)$ memory, whereas in this work, we study streaming algorithms with much smaller memory requirement of only $\text{polylog}{(n)}$ bits. This stringent memory requirement is in the same spirit of classical streaming algorithms that instead of recovering a full solution to the problem---which can be prohibitively large with such small memory as is the case in our problem---, aimed to learn certain statistical properties of their inputs. In our case, this translates to determining the ``(correlation) clusterability'' of input graphs, or more precisely, estimating the cost of the optimal correlation clustering solution. 

As our main result, we present two novel algorithms that in only $\text{polylog}{(n)}$ space are able to estimate the optimal correlation clustering cost up to some constant multiplicative factor plus some extra additive error. One of the algorithms outputs a $3$-multiplicative approximation plus $o(n^2)$ additive approximation, and the other one improves the additive error further down at the cost of increasing the multiplicative factor to some large constant. We then present new lower bounds that justify this mix of both multiplicative and additive error approximation in our algorithms.

Streaming Algorithms and Lower Bounds for Estimating Correlation Clustering Cost

Abstract

Correlation clustering is a fundamental optimization problem at the intersection of machine learning and theoretical computer science. Motivated by applications to big data processing, recent years have witnessed a flurry of results on this problem in the streaming model. In this model, the algorithm needs to process the input n -vertex graph by making one or few passes over the stream of its edges and using a limited memory, much smaller than the input size.

All previous work on streaming correlation clustering has focused on semistreaming algorithms with Ω( n ) memory, whereas in this work, we study streaming algorithms with much smaller memory requirements of only polylog ( n ) bits. This stringent memory requirement is in the same spirit of classical streaming algorithms that instead of recovering a full solution to the problem-which can be prohibitively large with such small memory as is the case in our problem-, aimed to learn certain statistical properties of their inputs. In our case, this translates to determining the '(correlation) clusterability' of input graphs, or more precisely, estimating the cost of the optimal correlation clustering solution.

As our main result, we present two novel algorithms that in only polylog ( n ) space are able to estimate the optimal correlation clustering cost up to some constant multiplicative factor plus some extra additive error. One of the algorithms outputs a 3 -multiplicative approximation plus o ( n 2 ) additive approximation, and the other one further reduces the additive error at the cost of increasing the multiplicative factor to some large constant. We then present new lower bounds that justify the mix of both multiplicative and additive error approximations in our algorithms.

1 Introduction

Correlation clustering is a fundamental optimization problem at the intersection of machine learning and theoretical computer science. This problem was introduced by the work of [BBC04] 1 , with motivation to document clustering, as follows: we have a complete graph G = ( V, E ) whose edges are labeled by either (+) or ( -) , and the objective, known as disagreement minimization , is to cluster the vertices so that the number of (+) edges across clusters and ( -) edges inside the same clusters are minimized. Correlation clustering has since found broad applications in areas such as document categorization [BBC04], webpage segmentation [CKP08], microscopy imaging [ZYH14], and community detection [VGW18, SDE + 21], to name a few.

There is an abundant body of literature studying polynomial time algorithms for correlation clustering. [BBC04, CMSY15] showed that there exists a 2 . 06 -approximation algorithm in polynomial time and

37th Conference on Neural Information Processing Systems (NeurIPS 2023).

the problem is NP-hard and even APX-hard. Furthermore, [ACN05] gave a simple (combinatorial) poly-time 3 -approximation algorithm that is widely used in practice. Very recently, breakthrough results by [CLN22, CALLN23] achieved 1 . 73 approximation in polynomial time. In addition, efficient algorithms are also explored for several variants of this problem like the agreement maximization objective [BBC04, Swa04, CGW03], weighted graphs [Swa04, CGW03, EF03], fixed number of clusters [GG06], fair clustering [AEKM20, SZ22], and others.

In recent years, with the rapid development of the 'big data era', there has been a growing interest in algorithms for correlation clustering under sublinear models. In general, learning algorithms under sublinear models are able to output the answer without processing or storing the entire input . For instance, [BGK13, AW22] gave algorithms to (approximately) learn the clustering in sublinear time, and [CLM + 21, BCMT22] designed algorithms in the Massively Parallel Computation (MPC) model. Another widely popular model of sublinear algorithms-the focus of our paper-is the graph streaming model . In this model, the edges of the input graph are given to the algorithm one by one in a stream and the memory of the algorithm is desired to be substantially smaller than the input. Here, [ACG + 15] designed a streaming algorithm with ˜ O ( n ) memory 2 that achieves 3 -approximation with O (log log( n )) passes - here, n is the number of vertices of the graph and thus the input size is Θ( n 2 ) bits. The number of passes of these algorithms were later improved to O (1) by [CLM + 21] and a single pass by [AW22], albeit with much larger, yet still a constant, approximation factor. Very recently, [BCMT23] further improved the approximation ratio of single-pass streaming algorithms to a 5 -approximation in polynomial time and (1 + ε ) -approximation in exponential-time.

The aforementioned line of work in [ACG + 15, CLM + 21, AW22, BCMT23] focused on the ˜ O ( n ) -memory regime, otherwise known as the semi-streaming memory. Allowing for Ω( n ) memory in these algorithms is necessary given that even outputting the solution, namely, the clustering of the input labeled graph, requires this much memory. Yet, in many application, Ω( n ) memory can still be quite large, and the implementations can require significant resource. In such cases, it is highly desirable to determine the 'clusterability' of the input graph before running the actual clustering algorithm. If even the optimal cost for the input is high, it implies the clustering cannot provide any meaningful outcome, and we should not waste resources on these instances. This type of 'value estimation' problem is extensively studied in the streaming literature, see e.g. [KKS15, GT19, ACL + 22, BOS22, DKPP22] for several examples in this context. This raises the following fundamental question:

How well can we estimate the optimal correlation clustering cost with polylog ( n ) -space streaming algorithms?

Despite the vast body of work on streaming correlation clustering in general, this question has received almost no attention so far. Indeed, to the best of our knowledge, the only prior work here is that of [ACG + 15] who proved that any (finite) purely multiplicative factor approximation of the cost is not possible in o ( n ) space. This result however does not rule out any additive error approximation (say, in the spirit of [BGK13] for local query algorithms).

We remedy this state of affairs in this paper. Our main algorithmic results show that one can obtain a 3 -approximation plus o ( n 2 ) additive approximation to the cost of optimal correlation clustering in only polylog ( n ) space - the additive approximation can be further reduced at the cost of increasing the multiplicative approximation to some large constant. We then complement these results with new streaming lower bounds that further justify the necessity of the additive errors in our algorithms. Throughout, we will present algorithms in the manner of insertion-only streams. However, we shall note that our algorithms can be extended to dynamic streams wherein the edges of the graph can be both inserted and deleted during the stream.

Our Contributions

To state our results, we need the following notation. Let OPT be the optimal cost for correlation clustering. We say an algorithm achieves an ( α, β ) -approximation of OPT if it gives an α -multiplicative with a β additive approximation, namely, outputs a number ALG such that

Our first main result is a single-pass streaming algorithm that achieves an ( O (1) , δn 2 ) -approximation with high probability in poly (log n, 1 /δ ) space.

Result 1. There is a single-pass streaming algorithm that outputs an ( O (1) , δn 2 ) -approximation of the optimal correlation clustering cost with high probability a and uses O ( polylog ( n ) /δ 5 ) space.

The δ -5 -dependence of our algorithm ensures that by setting δ = n -0 . 19 we can reduce the additive error down to n 1 . 81 = o ( n 2 ) and still achieve an o ( n ) -space algorithm. On the flip side however, the leading constant multiplicative factor is quite large in this algorithm - as we will see shortly, the algorithm is built on the sparse-dense decomposition idea from [AW22], which inevitably incurs a (worst-case) constant of at least 10 7 . Our next result addresses this drawback: we present another algorithm that achieves a (3 , δn 2 ) -approximation in expectation with polylog n space at the cost of an exponential dependence on δ in the space.

Result 2. There is a single-pass streaming algorithm that outputs a (3 , δn 2 ) -approximation of the optimal correlation clustering cost in expectation and uses 2 O (1 /δ ) · polylog ( n ) space.

Compared to the algorithm in Result 1, the algorithm in Result 2 is more suitable for instances whose optimal correlation clustering costs are large (e.g. OPT = Ω( n 2 ) ). In such a case, we can pick δ to be a small constant, and achieve a (3 + δ ) -multiplicative approximation with polylog ( n ) space.

Since both of our upper bounds contain the δn 2 additive error, we would naturally wonder to what extent these additive terms are necessary. We present two new lower bounds that partially address this question, and show that the additive errors are necessary to a large extent. Our first lower bound shows that if only additive error is allowed, there is no polylog ( n ) -space streaming algorithm in a single pass achieves additive error substantially better than O ( n 2 ) .

Result 3. No single-pass streaming algorithm with polylog ( n ) space can output a (1 , n 2 -ε ) approximation of the optimal correlation clustering cost with a sufficiently large constant probability of success for some ε = o (1) .

Result 3 can also be interpreted as the additive counterpart of the lower bounds in [ACG + 15], which focus instead on the purely multiplicative approximation. Since our upper bound allows both approximations, we further provide a second lower bound showing that a Θ( n ) additive error is necessary even if both multiplicative and additive errors are allowed. √

Result 4. No single-pass streaming algorithm with even o ( n ) space can output a (1 . 19 , O ( n )) approximation of the optimal correlation clustering cost with a sufficiently large constant probability of success.

We now discuss the applicable scenarios for our algorithms. To test instances with low vs. high correlation clustering costs (e.g. o ( n 2 ) vs. Ω( n 2 ) ), it suffices to run our algorithm in Result 1 with δ = 1 / log( n ) which uses ˜ O (1) space. On the other hand, to separate instances that are 'partially clusterable' (e.g. optimal cost of n 2 / 1000 ) vs. instances that are 'not clusterable at all' (e.g. optimal cost of n 2 / 5 ), it suffices to run our algorithm in Result 2 with δ = O (1) a small constant which uses ˜ O (1) space. The only case our algorithms are not able to deal with is to separate multiple 'well-clusterable' instances which does not seem that motivated in practice either.

Experiments To further validate our algorithms, we conduct experiments of our algorithms with the stochastic block model (SBM) extensively studied in the literature (e.g., [HLL83, ABH16, ZT23, Abb17a]). For correlation clustering, we can use a variate of the model that plants clusters with sizes Ω( n ) and samples (+) edges between the vertices in the same cluster with a large probability and ( -) edges with a low probability, and vice versa. Our implementation on the simulations of graph streams from the Stochastic Block Model shows that our algorithms consistently obtain approximations within a factor of 3 for the optimal clustering cost, while only storing 0 . 04% ∼ 3 . 6% 3 total edges when the graphs is moderately large. Furthermore, our algorithms are able to distinguish instances that are 'well-cluserable' vs. 'badly-clusterable' using a very small memory.

2 Preliminaries

We now introduce the notation and the problem definition. For standard technical tools, please see the supplementary material.

Notation. We use G = ( V, E + ∪ E -) to denote a labeled complete graph arriving in a stream. For any vertex v , we denote by N + ( v ) all vertices that have an (+) edge from v , and we let N + [ v ] = { v } ∪ N + ( v ) . Similarly, we denote by N -( v ) all vertices that have an ( -) edge from v , and we let N -[ v ] = { v } ∪ N -( v ) . We use N + ( u ) glyph[triangle] N + ( v ) (resp. N -( u ) glyph[triangle] N -( v ) ) to denote the disjoint neighborhoods of u and v , i.e. N + ( u ) glyph[triangle] N + ( v ) = ( N + ( u ) ∪ N + ( v )) \ ( N + ( u ) ∩ N + ( v )) . For a fixed set of vertices A ⊆ V , we further let E + ( v, A ) be the set of (+) edges between v and vertices in A , and E -( v, A ) be the set of ( -) edges between v and vertices in A .

We use G + = ( V, E + ) to denote the positive subgraph of G with all the (+) edges, and we define G -= ( V, E -) analogously. Note that since we work with labelled complete graph, the information of G -edges can be uniquely inferred from the positive subgraph G + . As such, when we work with G + only, we call a (+) edge ( u, v ) ∈ E + as an edge in G + . Similarly, we call a ( -) edge ( u, v ) ∈ E -an non-edge in G + . We may omit G + when the context is clear.

For a fixed cluster C on a labeled complete graph G , we use cost ( C ) to denote the total cost of correlation clustering on G by C (we omit G since it is obvious by the context). Furthermore, we slightly abuse the notation to reload cost ( · ) as a function of cost on subgraphs of G in the following occasions: 1) For an induced subgraph H ⊆ G , we let cost ( C , H ) be the cost of C induced by the edges whose both endpoints are in H ; 2) For two vertex sets A ∪ B ⊆ V , we let cost ( C , ( A,B )) be the cost of C induced by the edges with exactly one endpoint in A and one endpoint in B .

Finally, for a single edge ( u, v ) , we use the notation edge-cost ( C , ( u, v )) to denote the cost induced by a single edge ( u, v ) for clustering C . We may write a short-hand notation edge-cost ( ( u, v )) when the context is clear that C is used.

Problem Definition

We now give the formal description of the problem.

Problem 1 (Correlation Clustering Value Estimation) . Given a labeled complete graph G = ( V, E -∪ E + ) and a clustering C that partitions V into disjoint set of vertices C 1 , C 2 , · · · , C k , the cost of disagreement minimization correlation clustering on C is defined as glyph[negationslash]

Let OPT be the minimum cost over all possible clusterings. The Correlation Clustering Value Estimation problem asks for a number ALG such that OPT ≤ ALG ≤ f ( OPT ) for some function f ( x ) ≥ x . If f ( OPT ) = α · OPT + β · n 2 for α and β , we say that ALG is a ( α, β ) -approximation of the value of correlation clustering in this scenario.

We study algorithms for correlation clustering value estimation under the graph streaming model, where edges arrive one after another in a stream with the labels. We give a more formal definition of the streaming model (and the more general dynamic streaming model) in the supplementary material.

3 An Algorithm based on Sparse-dense Decomposition

In this section, we present our first streaming algorithm that achieves a ( O (1) , δn 2 ) -approximation for testing the value of correlation clustering in O ( polylog ( n ) / poly ( δ )) space, as long as δ ≥ 1 / polylog ( n ) . We utilize the idea in [AW22] to test a ( O (1) , δn 2 ) -approximate value of the sparsedense decomposition-based correlation clustering cost, which in turn is an O (1) approximation of the optimal cost. Our algorithm uses a memory of O ( polylog ( n ) / poly ( δ )) bits, which is efficient for large-scale inputs. More formally, the guarantee of our algorithm is as follows.

Theorem 1. There is a (dynamic) streaming algorithm that with high probability gives a ( O (1) , δn 2 ) -approximation for the correlation clustering value and take space O ( log 2 ( n ) δ 5 ) words.

On a high level, our algorithm for Theorem 1 uses the idea to approximate the value of optimal correlation clustering with sparse-dense decomposition (see supplementary material). It is shown in

[AW22] that once we can find such a decomposition on G + , we can achieve an O (1) -approximation by simply putting every sparse vertex into a singleton cluster and gathering each almost-clique K i in a separate cluster. Therefore, an approximation of the number of the aforementioned edges in a fixed decomposition will result in a good estimation of the cost.

However, since the sparse-dense decomposition is not unique, it is unclear how to estimate the edges for a fixed decomposition. On the other hand, an algorithm with polylog ( n ) space is necessarily oblivious of the decomposition since it takes Θ( n ) bits to write it down. To overcome the problem, we forgo the strict notion of the sparse-dense decomposition, and utilize the notion of ε -sparse edges and ε -dense non-edges (of G + ) instead. We now formally define these notions as follows.

Definition 1. Fix an arbitrary graph G = ( V, E ) (which is not necessarily labeled and complete) and a vertex pair ( u, v ) ∈ G , we say

( u, v ) is an ε -sparse edge (resp. non-edge) if ( u, v ) ∈ E (resp. ( u, v ) glyph[negationslash]∈ E ) and | N ( v ) glyph[triangle] N ( u ) | ≥ ε · max { deg ( u ) , deg ( v ) } .

( u, v ) is an ε -dense edge (resp. non-edge) if ( u, v ) ∈ E (resp. ( u, v ) glyph[negationslash]∈ E ) and | N ( v ) glyph[triangle] N ( u ) | ≤ ε · max { deg ( u ) , deg ( v ) } .

Note that the definitions of the ε -sparse and ε -dense edges/non-edges are generic and not restricted to the labeled complete graph (or even the correlation clustering application). Similarly, the sketching tools we design in this section are also generic: we will use them in the context of correlation clustering later. We now prove the existence of the following tools.

Lemma 3.1. There exists a dynamic streaming algorithm Tool-spr with parameters ε, δ that given any graph G = ( V, E ) in a stream and a pair of vertices u, v ∈ V , satisfying the promise deg + ( u ) ≥ δn and deg + ( v ) ≥ δn , with high probability outputs 'Yes" if ( u, v ) is at least ε -sparse and 'No" if ( u, v ) is not ε 8 -sparse using O (1 /ε 2 δ ) words of space.

Lemma 3.2. There exists a dynamic streaming algorithm Tool-dns with parameters ε, δ that given any graph G = ( V, E ) in a stream and a pair of vertices u, v ∈ V , satisfying the promise deg + ( u ) ≥ δn and deg + ( v ) ≥ δn , with high probability outputs 'Yes" if ( u, v ) is at most ε -dense and 'No" if ( u, v ) is not 8 ε -dense using O (1 /ε 2 δ ) words of space.

We defer the proofs of Lemmas 3.1 and 3.2 to the supplementary material, and use them as blackboxes in the rest of the paper. We now discuss how to use the ε -sparse edges and ε -dense non-edges (defined in Definition 1) in G + to estimate the correlation clustering cost. In what follows, we use E + ε -sparse to denote the set of ε -sparse (positive) edges, and E -ε -dense to denote the set of ε -dense non-edges. Furthermore, for each vertex v , we use E -ε -dense , v to denote the set of ε -dense non-edges incident on v . We let m + ε -sparse := ∣ ∣ E + ε -sparse ∣ ∣ denote the number of ε -sparse edges, m -ε -dense := ∣ ∣ E -ε -dense ∣ ∣ denote the number of ε -dense non-edges and let m -ε -dense ,v := ∣ ∣ ∣ E -ε -dense , v ∣ ∣ ∣ denote the number of ε -dense non-edges incident on v . Finally, we define ̂ m -ε -dense as follows.

Definition 2. For each vertex v define ̂ m -ε -dense ,v := min { m -ε -dense ,v , deg + ( v ) } . Furthermore, let ̂ m -ε -dense := ∑ v ∈ V ̂ m -ε -dense ,v .

The intuition behind ̂ m -ε -dense ,v is to count the non-edges (of G + ) in E -ε -dense , v for at most deg + ( v ) times . Our estimator for ε -dense non-edges will estimate ̂ m -ε -dense instead of m -ε -dense for the following reason: it suffices to estimate ̂ m -ε -dense since the number of non-edges inside each almost-clique is at most deg + ( v ) ; on the other hand, if we estimate all ε -dense non-edges, there could be a very large overhead since the non-edges between almost-cliques are also counted. We note that if ̂ m -ε -dense ,v = m -ε -dense ,v for all v ∈ V then ̂ m -ε -dense is twice m -ε -dense (since we are double counting edges). But this is a 2 -approximation in the worst case, and we do this to make calculations easier.

We prove the following lemmas that establish the connections between the aforementioned sets of edges and the edges from the sparse-dense decomposition. Due to space limits, we directly present the properties of our estimators, and defer the proof to the supplementary material.

Lemma 3.3. Suppose G = ( V, E ) is any labeled graph and V = V sparse glyph[unionsq] K 1 glyph[unionsq] . . . glyph[unionsq] K k is an ε -sparse-dense decomposition of G + for 0 ≤ ε ≤ 1 / 360 and η 0 ≤ 1 / 20 . Let C SDD be the cost of

correlation clustering when the clusters are the almost cliques and sparse vertices are in singleton clusters. Then we have 2 η 0 · ε · m + η 0 ε -sparse + m + ε -sparse + ̂ m -4 ε -dense ≥ C SDD ≥ OPT .

We upper bound the cost of m + ε -sparse and ̂ m -ε -dense by a function of OPT using a charging argument.

Lemma 3.4. Suppose G = ( V, E ) is any labeled graph and OPT be the optimal correlation clustering cost, and let ε ≤ 1 32 and β ≤ 1 2 ε , then m + βε -sparse ≤ 2 βε · OPT and ̂ m -βε -dense ≤ 8 · OPT .

By Lemmas 3.3 and 3.4 if we are able to exactly recover m + ε -sparse and ̂ m -ε -dense then we would get an O (1) approximation of OPT . Such a task seems difficult, however, using Lemmas 3.1 and 3.2 we design our tools for estimating m + ε -sparse and ̂ m -ε -dense with an additive error.

Lemma 3.5. There exists a dynamic streaming algorithm with parameters ε, δ that given any graph G = ( V, E ) in a stream returns a value Z sp that is at least m + ε -sparse and at most m + ε/ 8 -sparse + δn 2 with high probability and uses O (log n/ε 2 δ 3 ) words of space.

Lemma 3.6. There exists a dynamic streaming algorithm with parameters ε, δ that given any graph G = ( V, E ) in a stream returns a value Z den that is at least ̂ m -ε -dense and at most ̂ m -8 ε -dense + δn 2 with high probability and uses O (log 2 n/ε 2 δ 5 ) words of space.

Finalizing the proof sketch of Theorem 1. We are given a parameter δ as input and we want the additive error to be at most δn 2 . We fix ε = 1 / 360 , η 0 = 1 / 20 and δ ′ = δ · (2 + 2 η 0 ε ) -1 .

Run the algorithm in Lemma 3.5 with parameters ε and δ ′ , and let the output be Z ε sp . Also, run the algorithm in Lemma 3.5 with parameters η 0 ε and δ ′ , and let the output be Z η 0 ε sp . Run the algorithm in Lemma 3.6 with parameters 4 ε and δ ′ , and let the output be Z den .

Lemmas 3.3 to 3.6 imply that the cost Z CC we output satisfies:

The space taken is O (log n/δ 3 ) for both copies of Lemma 3.5 and O (log 2 n/δ 5 ) for Lemma 3.6 giving a total space of O (log 2 n/δ 5 ) words. This proves Theorem 1.

4 An Algorithm based on Pivot

In this section, we give our second streaming algorithm that is a (3 + γ, δn 2 ) -approximation for the correlation clustering value for any choice of δ and γ < 1 / 2 . The algorithm works in O ( polylog ( n )) space as long as δ ≥ Ω( 1 log log( n ) ) . Consider the following formal statement:

Theorem 2. There is a (dynamic) streaming algorithm that with high probability gives a (3+ γ, δn 2 ) -approximation for the correlation clustering value and takes space O ( 2 7 / 6 δ · log 2 n γ · δ 5 ) words.

Our algorithm in Theorem 2 is inspired by the Local Cluster algorithm from [BGK13]. The Local Cluster algorithm samples 1 /δ random vertices in a set U and computes the greedy maximal independent set (MIS) M of U to get the cluster centers p 1 , p 2 , . . . , p t . The clusters generated then are N + [ p i ] -∪ i -1 j =1 N + [ p j ] and all the remaining vertices (called unclustered vertices) are clustered in their own singleton cluster. [BGK13] proved that the expected cost of this clustering is at most 3 · OPT + δ 2 n 2 . More formally, they showed the following:

Proposition 4.1 ([BGK13]) . The expected cost of Local Cluster is at most 3 · OPT + δ 2 n 2 .

To simulate the Local Cluster algorithm in a streaming manner, while forming a cluster we need to know which of its neighbors are already clustered in previous clusters which requires new sketching tools (on G + ) that are different from the ones we used in Section 3. These tools estimate the number of non-edges within or the number of edges going out of the neighborhood of a vertex ( u ) outside of the (+) neighborhood of a known set of vertices ( S ). In other words, fix a graph G = ( V, E ) , a vertex u , and a set S , we want to estimate the number of non-edges within or the number of edges going out of N [ u ] -N [ S ] . We again use the generic form to present the sketching tools and do not specify (+) edges. The formal definitions can be given as follows.

Definition 3. A non-edge ( x, y ) is within N [ u ] -N [ S ] iff: i ) x glyph[negationslash]∈ N [ S ] and ii ) y glyph[negationslash]∈ N [ S ] and iii ) x ∈ N [ u ] and y ∈ N [ u ] .

Definition 4. An edge ( x, y ) is going out of N [ u ] -N [ S ] iff i ) x glyph[negationslash]∈ N [ S ] and ii ) y glyph[negationslash]∈ N [ S ] and iii ) x ∈ N [ u ] or y ∈ N [ u ] but not both.

Definition 5. An edge ( x, y ) is unclustered w.r.t. S iff i ) x glyph[negationslash]∈ N [ S ] and ii ) y glyph[negationslash]∈ N [ S ] .

We also define m ne ( u, S ) as the number of non-edges within N [ u ] -N [ S ] , m e ( u, S ) as the number of edges going out of N [ u ] -N [ S ] , and m u ( S ) as the number of unclustered edges w.r.t. S . We write m ne , m e , and m u when u and S are clear from context.

Exactly recovering m u , m ne and m e is difficult, so we have the following lemma about estimating the number of non-edges within N [ u ] -N [ S ] , the edges going out of N [ u ] -N [ S ] , and the unclustered edges with respect to S :

Lemma 4.2. There exist streaming algorithms called NE-Tool , E-Tool , and U-Tool that given u and S before the stream, respectively compute with high probability i ) . the number of non-edges within ; ii ) . the number of edges going out of N [ u ] -N [ S ] ; and iii ) . the number of edges with unclustered w.r.t. S with an overestimation of at most δn 2 and take space O (1 /δ 2 ) words.

Using the tools we have built so far, we present a (3 , δn 2 ) approximation algorithm in expectation.

Algorithm 1. Simulation of the Local Cluster algorithm

Input: + - 2

G = ( V, E ∪ E ) in a (dynamic) stream

Output: (3 , δn ) -approximation to the correlation clustering value in expectation

Pre-Processing:

Sample a set U of 1 /δ random vertices

Let π be a random permutation of the vertices in U

During the Stream:

Store (+) all edges between vertices of U

For all u ∈ U and S ⊆ U compute NE-Tool ( u, S ) , E-Tool ( u, S ) , U-Tool ( S ) with parameter δ 2 / 6 using the G + subgraph.

Post-Processing:

Compute the greedy MIS M := p 1 , p 2 , . . . , p t of U in the order of π (using edges stored between vertices of U ).

S 0 = ∅ . ˜ Z = 0 .

For i = 1 to t :

˜ Z = ˜ Z + NE-Tool ( p i , S i -1 ) + E-Tool ( p i , S i -1 ) .

S i := S i -1 ∪ { p i }

Output Z := ˜ Z + U-Tool ( M )

Algorithm 1 estimates m u , m ne and m e with at most δn 2 additive error. We now show that with the exact values of m u , m ne and m e , we get the exact clustering value of the Local Cluster algorithm.

Claim 4.3. The clustering value of Local Cluster is equal to m u + ∑ i m ne ( p i , S i -1 )+ m e ( p i , S i -1 ) .

Combining Lemma 4.2 and Claim 4.3 gives us the following performance guarantees for Algorithm 1.

Lemma 4.4. The output Z of Algorithm 1 is at least OPT with high probability and is at most 3 OPT + δn 2 in expectation.

Proof. Using Claim 4.3 we know that if the tools worked with no error, Algorithm 1 would give the exact clustering cost of Local Cluster. Also, we know that the tools do not underestimate and overestimate by δ 2 n 2 / 6 with high probability (Lemma 4.2). We first note that Z is at least OPT because of the above conditioning and the fact that the clustering cost of Local Cluster is at least OPT . Using Proposition 4.1 we know that the expected cost of the clustering when choosing 1 δ random pivots is at most 3 OPT + δ 2 n 2 . Therefore, the clustering cost of Algorithm 1 is between OPT and 3 OPT + δ 2 n 2 plus the overestimate.

We now calculate the overestimate. We use parameter δ 2 / 6 for the tools thus the additive error in each tool is at most δ 2 n 2 / 6 . There are 1 /δ pivots implying a total additive error of at most δn 2 / 3 over all the copies of NE-Tool and E-Tool . U-Tool has an error of at most δ 2 n 2 / 6 implying that the overall error is at most δn 2 / 2 + δn 2 / 3 + δ 2 n 2 / 6 ≤ δn 2 giving a (3 , δn 2 ) approximation in expectation.

We note that we condition on the high probability events for all copies of the tools and union bound over the failure probabilities. Thus, for the overall failure probability to be small we need 2 1 /δ ≤ poly ( n ) . This requirement is trivially statisfied by the fact that 2 1 /δ is on the space bound (Lemma 4.5), and the space bound for any streaming algorithm is at most O ( n 2 ) .

We now show the space bound of Algorithm 1.

Lemma 4.5. The space of Algorithm 1 is O ( 2 1 /δ · log n δ 5 ) words.

Proof. Each copy of the tools with parameter δ 2 / 6 takes O (log n/δ 4 ) words of space, and we compute the tools for all v ∈ U and S ⊆ U . Thus, the space used is O (2 1 /δ · log n/δ 5 ) words. Storing edges between vertices in U takes space at most 1 /δ 2 words which is a lower order term.

Lemmas 4.4 and 4.5 together give us a (3 , δn 2 ) approximation in expectation using O ( 2 1 /δ · log n δ 5 ) space. We now prove Theorem 2. We run Algorithm 1 60 log n γ times in parallel and let Z min be the minimum cost over all iterations. Z min is a (3 + γ, 7 6 δn 2 ) -approximation with high probability.

Claim 4.6. OPT ≤ Z min ≤ (3 + γ ) OPT + 7 6 δn 2 with high probability.

Thus, we get a (3 + γ, 7 6 δn 2 ) -approximation with high probability. Each parallel repetition of the algorithm takes O ( 2 1 /δ · log n δ 5 ) words of space. Repeating O ( log n γ ) times and re-scaling δ by a factor of 7 6 gives a total space bound of O ( 2 7 / 6 δ · log 2 n γ · δ 5 ) words. This completes the proof of Theorem 2.

5 A Lower Bound for O ( n 2 -ε ) Additive error

In this section, we show that if we only allow additive error, any streaming algorithm with polylogarithm memory cannot cross an error barrier of Ω( n 2 -ε ) for some ε = o (1) . Here, and throughout, we will refer this lower bound as the almost-quadratic lower bound . The lower bound is weaker than the linear lower bound of Section 6 in terms of the multiplicative factor since it only works for c = 1 . However, it is much stronger in the additive sense: the upper bounds obtained by our algorithms are O ( n 2 ) , and the almost-quadratic lower bound matches this term up to an O ( n ε ) factor - this provides a strong justification of the additive error in our algorithms.

The formal statement of the almost-quadratic lower bound is as follows.

Theorem 3. There exists a constant C , such that any single-pass streaming algorithm that estimates the optimal value OPT of correlation clustering by a C · n 2 -ε purely additive error (i.e., an estimated value that is at most ( OPT + C · n 2 -ε ) ) with probability at least 99 100 has to use a memory of Ω( n ε ) bits, even on labeled complete graphs.

Note that Theorem 3 does not require the stream to be dynamic, which is in contrast to our upper bound results that work for dynamic streams. We obtain the almost-quadratic lower bound by a new reduction from the INDEX problem. On the high level, the instance we construct 'hides' an Ω( n 2 -ε ) gap between the yes and no cases inside a case-invariant Ω ( n 2 ) cost. The reduction can be viewed as a more involved variant of the space lower bound for the exact correlation clustering in a very recent work [AAD + 23]. In a nutshell, we modify their construction to 'boost' the gap between yes and no cases, and apply a new trick to separate the values of clustering. Due to space limits, we defer the proof of Theorem 3 to the supplementary material.

6 A Lower Bound for O ( n ) Additive error

In this section, we show that any dynamic streaming algorithm that gets a ( c, εn ) -approximation for c < 1 . 2 and O ( n ) additive error needs Ω( √ n ) bits of space. Here, and throughout, we will call this lower bound the linear lower bound . Formally, we have:

Theorem 4. Let c ∈ [1 , 6 5 ) and ε ∈ (0 , 6 5 -c ) . Any single-pass streaming algorithm that estimates the optimal value OPT of correlation clustering by a ( c, εn ) -approximation (i.e., an estimated value that is at most ( c · OPT + ε · n ) ) with probability at least 99 100 has to use a memory of Ω( √ n ) bits, even on labeled (complete) graphs.

Similar to the case in Theorem 3, Theorem 4 does not require the stream to be dynamic. Comparing with the almost-quadratic lower bound we show in Section 5, the lower bound in Theorem 4 is weaker in the additive sense. However, it allows the multiplicative approximation of the algorithm to be > 1 , while the lower bound in Section 5 only rules out algorithms with purely additive errors.

Our lower bound uses the celebrated machinery from Boolean Hidden Hypermatching (BHH) and Gap Cycle Counting (GCC) pioneered by [VY11]. The Gap Cycle Counting (GCC) lower bound states that for any algorithm to distinguish whether a graph consists of cycles with length 2 t or cycles with length 4 t for some t ≥ 2 , a memory of Ω ( n 1 -1 t ) bits is necessary. On the high level, our plan is to show that for graphs similar to the ones prescribed in the GCC problem, the values of correlation clustering are different by an additive gap of O ( n ) . Therefore, by a reduction argument, any algorithm that breaks this barrier of additive gap requires ω ( polylog ( n )) memory.

Due to space limits, we defer the lower bound construction to the supplementary material.

7 Experiments

We describe in this section the experiments of algorithms on the Stochastic Block Model and the Erdos-Renyi random graphs. These experiments show that for a very natural family of graphs, our algorithms can achieve a very competitive performance on the estimated cost values, and the performances are often much better than the worst-case theoretical analysis. Furthermore, our algorithms are capable of separating 'well-clusterable' vs. 'badly-clusterable' instances.

Experiment Settings Limited by space, we sketch the key settings of our experiment, and defer the full discussion to the full version in the supplementary material. As we have discussed, we perform our experiments on the data generated from the well-studied Stochastic Block Model (SBM) that plants ground-truth clusters with sizes Ω( n ) , samples (+) edge between vertex pairs ( x i , x j ) in the same planted cluster with probability p > 0 . 5 , and samples (+) edges between vertex pairs ( x i , x j ) in different clusters with probability 1 -p . The SBM captures a lot of real-world scenarios, including social networks [HLL83], community detection [Abb17b], graph clustering [LW19], and Bioinformatics [MGC21], to name a few.

We test SBM instances that are reasonably 'cluserable', i.e., we set p = 0 . 8 with vertices n = 500 , n = 1000 , and n = 2000 . Furthermore, we compare the costs estimated by algorithms for the SBM and the Erdos-Renyi random graphs G ( n, p ) with p = 0 . 5 . In this regime, the Erdos-Renyi graph does not appear to have any clustering property, and the cost is high. Under the n = 1000 setting, we tested whether our algorithms can distinguish the costs between 'good' (SBM with p = 0 . 95 ) and 'bad' (Erdos-Renyi with p = 0 . 5 ) instances - a property that can be extremely useful in practice.

We implement our algorithms based on our descriptions in Section 3 and Section 4 with some relaxation of parameters. Notably, for the pivot-based algorithm, we slightly relax the requirement to allow 2-pass over the stream; as such, we can use the first pass to perform greedy MIS on the sampled vertex set, so we do not have to pay the 2 O (1 /δ ) factor in the space.

We evaluate the performances mainly based on two metrics: the multiplicative factor of the cost estimation (which we call the 'competitive ratio') and the fraction of the edges used. To overcome the possible effects of random seed, we fix our random seed from 0 to 14 , and run 15 experiments. We include the error bars and the curves of the competitive ratios and fraction of the edges. For the experiment to distinguish SBM and Erdos-Renyi graphs, we simply plot the two types of costs w.r.t. experiment runs, and give the distributions of the costs. All the experiments were conducted on two macbooks and public Colab clusters.

Experimental Results We first show that our algorithms are insensitive to the choice of parameters - this property is evident by the figures we used in the full version in the supplementary material. As such, we focus on the settings with ε = 0 . 01 for the SDD-based algorithm and δ = 0 . 1 for the pivot-based algorithm for the rest of this section.

Due to the space limit, we only discuss our algorithm based on sparse dense decomposition (SDD). The left 3 plots in Figure 1 give the plots of the cost estimation for the SDD-based algorithm on n = 2000 . The figures show that the approximation factor for this algorithm is roughly between 1 and 2 , our algorithm consistently uses less than 4% of the edges in the graph.

The results for our SDD-based algorithm to distinguish between SBM instances with p = 0 . 95 and ER instances with p = 0 . 5 with the SDD-based algorithm is shown in the right two plots of Figure 1. From the figure, it can be observed that the the SDD-based algorithm outputs drastically different clustering costs of the SBM instances vs. the Erdos-Renyi instances. We can observe from the left plot that the supports of the costs are disjoint . As such, by a simple linear threshold, our SDD algorithm is able to perfectly distinguish between both types of instances while using less than 10% of the edges ( n = 1000 case).

800000

1.8

0.036

700000

1.7

600000

1.6

0.034

500000

1.5

400000

1.4

0.032

300000

1.3

200000

1.2

0.030

100000

1.1

0.028

SDD-alg Estimation

Real Cost

123456789101112131415

1234567891011121314 15

BoxPlot of the Costs

Multi-factor of the Esimation Cost

Fraction of edges

4.0

Erdos-Renyi

SBM

300000

3.5

3.0

250000

2.5

2.0

1.5

1.0

100000

0.5

50000

Erdos-Renyi

SBM

0.0

123

4

5

678910

1112131415

SDD-Alg Cost Estimation (log scale)

Index of Experiment

Acknowledgments and Disclosure of Funding

Research supported in part by the Alfred P. Sloan Research Fellowship, a Faculty of Mathematics Research Chair Award from University of Waterloo, an NSF CAREER Grant CCF-2047061, a gift from Google Research, and a Rutgers Research Council Fulcrum Award.

References

[AAD + 23] Vikrant Ashvinkumar, Sepehr Assadi, Chengyuan Deng, Jie Gao, and Chen Wang. Evaluating stability in massive social networks: Efficient streaming algorithms for structural balance. In Nicole Megow and Adam D. Smith, editors, Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, APPROX/RANDOM 2023, September 11-13, 2023, Atlanta, Georgia, USA , volume 275 of LIPIcs , pages 58:1-58:23. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2023.

[Abb17a] Emmanuel Abbe. Community detection and stochastic block models: Recent developments. J. Mach. Learn. Res. , 18:177:1-177:86, 2017.

[Abb17b] Emmanuel Abbe. Community detection and stochastic block models: recent developments. The Journal of Machine Learning Research , 18(1):6446-6531, 2017.

[ABH16] Emmanuel Abbe, Afonso S. Bandeira, and Georgina Hall. Exact recovery in the stochastic block model. IEEE Trans. Inf. Theory , 62(1):471-487, 2016.

[ACG + 15] Kook Jin Ahn, Graham Cormode, Sudipto Guha, Andrew McGregor, and Anthony Wirth. Correlation clustering in data streams. In Francis R. Bach and David M. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015 , volume 37 of JMLR Workshop and Conference Proceedings , pages 2237-2246. JMLR.org, 2015.

[ACL + 22] Sepehr Assadi, Vaggos Chatziafratis, Jakub Lacki, Vahab Mirrokni, and Chen Wang. Hierarchical clustering in graph streams: Single-pass algorithms and space lower bounds. In Po-Ling Loh and Maxim Raginsky, editors, Conference on Learning Theory, 2-5 July 2022, London, UK , volume 178 of Proceedings of Machine Learning Research , pages 4643-4702. PMLR, 2022.

[ACN05] Nir Ailon, Moses Charikar, and Alantha Newman. Aggregating inconsistent information: ranking and clustering. In Harold N. Gabow and Ronald Fagin, editors, Proceedings of the 37th Annual ACM Symposium on Theory of Computing, Baltimore, MD, USA, May 22-24, 2005 , pages 684-693. ACM, 2005.

[AEKM20] Sara Ahmadian, Alessandro Epasto, Ravi Kumar, and Mohammad Mahdian. Fair correlation clustering. In Silvia Chiappa and Roberto Calandra, editors, The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy] , volume 108 of Proceedings of Machine Learning Research , pages 4195-4205. PMLR, 2020.

[AW22] Sepehr Assadi and Chen Wang. Sublinear time and space algorithms for correlation clustering via sparse-dense decompositions. In Mark Braverman, editor, 13th Innovations in Theoretical Computer Science Conference, ITCS 2022, January 31 - February 3, 2022, Berkeley, CA, USA , volume 215 of LIPIcs , pages 10:1-10:20. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.

[BBC04] Nikhil Bansal, Avrim Blum, and Shuchi Chawla. Correlation clustering. Mach. Learn. , 56(1-3):89-113, 2004.

[BCMT22] Soheil Behnezhad, Moses Charikar, Weiyun Ma, and Li-Yang Tan. Almost 3approximate correlation clustering in constant rounds. In 63rd IEEE Annual Symposium on Foundations of Computer Science, FOCS 2022, Denver, CO, USA, October 31 November 3, 2022 , pages 720-731. IEEE, 2022.

[BCMT23] Soheil Behnezhad, Moses Charikar, Weiyun Ma, and Li-Yang Tan. Single-pass streaming algorithms for correlation clustering. In Proceedings of the 34th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2023 (to appear) , 2023.

[BGK13] Francesco Bonchi, David García-Soriano, and Konstantin Kutzkov. Local correlation clustering. CoRR , abs/1312.5105, 2013.

[BOS22] Tomer Berg, Or Ordentlich, and Ofer Shayevitz. On the memory complexity of uniformity testing. In Po-Ling Loh and Maxim Raginsky, editors, Conference on Learning Theory, 2-5 July 2022, London, UK , volume 178 of Proceedings of Machine Learning Research , pages 3506-3523. PMLR, 2022.

[CALLN23] Vincent Cohen-Addad, Euiwoong Lee, Shi Li, and Alantha Newman. Handling correlated rounding error via preclustering: A 1.73-approximation for correlation clustering, 2023.

[CGW03] Moses Charikar, Venkatesan Guruswami, and Anthony Wirth. Clustering with qualitative information. In 44th Symposium on Foundations of Computer Science (FOCS 2003), 11-14 October 2003, Cambridge, MA, USA, Proceedings , pages 524-533. IEEE Computer Society, 2003.

[CKP08] Deepayan Chakrabarti, Ravi Kumar, and Kunal Punera. A graph-theoretic approach to webpage segmentation. In Jinpeng Huai, Robin Chen, Hsiao-Wuen Hon, Yunhao Liu, Wei-Ying Ma, Andrew Tomkins, and Xiaodong Zhang, editors, Proceedings of the 17th International Conference on World Wide Web, WWW 2008, Beijing, China, April 21-25, 2008 , pages 377-386. ACM, 2008.

[CLM + 21] Vincent Cohen-Addad, Silvio Lattanzi, Slobodan Mitrovic, Ashkan Norouzi-Fard, Nikos Parotsidis, and Jakub Tarnawski. Correlation clustering in constant many parallel rounds. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , volume 139 of Proceedings of Machine Learning Research , pages 2069-2078. PMLR, 2021.

[CLN22] Vincent Cohen-Addad, Euiwoong Lee, and Alantha Newman. Correlation clustering with sherali-adams. In 63rd IEEE Annual Symposium on Foundations of Computer Science, FOCS 2022, Denver, CO, USA, October 31 - November 3, 2022 , pages 651661. IEEE, 2022.

[CMSY15] Shuchi Chawla, Konstantin Makarychev, Tselil Schramm, and Grigory Yaroslavtsev. Near optimal LP rounding algorithm for correlationclustering on complete and complete k-partite graphs. In Rocco A. Servedio and Ronitt Rubinfeld, editors, Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, STOC 2015, Portland, OR, USA, June 14-17, 2015 , pages 219-228. ACM, 2015.

[DKPP22] Ilias Diakonikolas, Daniel M. Kane, Ankit Pensia, and Thanasis Pittas. Streaming algorithms for high-dimensional robust statistics. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , volume 162 of Proceedings of Machine Learning Research , pages 5061-5117. PMLR, 2022.

[EF03] Dotan Emanuel and Amos Fiat. Correlation clustering - minimizing disagreements on arbitrary weighted graphs. In Giuseppe Di Battista and Uri Zwick, editors, Algorithms ESA 2003, 11th Annual European Symposium, Budapest, Hungary, September 16-19, 2003, Proceedings , volume 2832 of Lecture Notes in Computer Science , pages 208-220. Springer, 2003.

[GG06] Ioannis Giotis and Venkatesan Guruswami. Correlation clustering with a fixed number of clusters. Theory Comput. , 2(13):249-266, 2006.

[GT19] Venkatesan Guruswami and Runzhou Tao. Streaming hardness of unique games. In Dimitris Achlioptas and László A. Végh, editors, Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, APPROX/RANDOM 2019, September 20-22, 2019, Massachusetts Institute of Technology, Cambridge, MA, USA , volume 145 of LIPIcs , pages 5:1-5:12. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2019.

[GW89] Martin Grötschel and Yoshiko Wakabayashi. A cutting plane algorithm for a clustering problem. Math. Program. , 45(1-3):59-96, 1989.

[HLL83] Paul W Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social networks , 5(2):109-137, 1983.

[KKS15] Michael Kapralov, Sanjeev Khanna, and Madhu Sudan. Streaming lower bounds for approximating MAX-CUT. In Piotr Indyk, editor, Proceedings of the Twenty-Sixth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2015, San Diego, CA, USA, January 4-6, 2015 , pages 1263-1282. SIAM, 2015.

[LW19] Clement Lee and Darren J Wilkinson. A review of stochastic block models and extensions for graph clustering. Applied Network Science , 4(1):1-50, 2019.

[MGC21] Leonardo Morelli, Valentina Giansanti, and Davide Cittaro. Nested stochastic block models applied to the analysis of single cell data. BMC bioinformatics , 22(1):1-19, 2021.

[SDE + 21] Jessica Shi, Laxman Dhulipala, David Eisenstat, Jakub Lacki, and Vahab S. Mirrokni. Scalable community detection via parallel correlation clustering. Proc. VLDB Endow. , 14(11):2305-2313, 2021.

[Swa04] Chaitanya Swamy. Correlation clustering: maximizing agreements via semidefinite programming. In J. Ian Munro, editor, Proceedings of the Fifteenth Annual ACMSIAM Symposium on Discrete Algorithms, SODA 2004, New Orleans, Louisiana, USA, January 11-14, 2004 , pages 526-527. SIAM, 2004.

[SZ22] Roy Schwartz and Roded Zats. Fair correlation clustering in general graphs. In Amit Chakrabarti and Chaitanya Swamy, editors, Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, APPROX/RANDOM 2022, September 19-21, 2022, University of Illinois, Urbana-Champaign, USA (Virtual Conference) , volume 245 of LIPIcs , pages 37:1-37:19. Schloss Dagstuhl - LeibnizZentrum für Informatik, 2022.

[VGW18] Nate Veldt, David F. Gleich, and Anthony Wirth. A correlation clustering framework for community detection. In Pierre-Antoine Champin, Fabien Gandon, Mounia Lalmas, and Panagiotis G. Ipeirotis, editors, Proceedings of the 2018 World Wide Web Conference on World Wide Web, WWW 2018, Lyon, France, April 23-27, 2018 , pages 439-448. ACM, 2018.

[VY11] Elad Verbin and Wei Yu. The streaming complexity of cycle counting, sorting by reversals, and other problems. In Dana Randall, editor, Proceedings of the TwentySecond Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2011, San Francisco, California, USA, January 23-25, 2011 , pages 11-25. SIAM, 2011.

[ZT23] Qiaosheng Eric Zhang and Vincent Y. F. Tan. Exact recovery in the general hypergraph stochastic block model. IEEE Trans. Inf. Theory , 69(1):453-471, 2023.

[ZYH14] Chong Zhang, Julian Yarkony, and Fred A. Hamprecht. Cell detection and segmentation using correlation clustering. In Polina Golland, Nobuhiko Hata, Christian Barillot, Joachim Hornegger, and Robert D. Howe, editors, Medical Image Computing and Computer-Assisted Intervention - MICCAI 2014 - 17th International Conference, Boston, MA, USA, September 14-18, 2014, Proceedings, Part I , volume 8673 of Lecture Notes in Computer Science , pages 9-16. Springer, 2014.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies correlation clustering in the streaming model where the edges of the underlying graph are updated one  at a time. Where previous work has focused on the semi streaming model with $\\Omega(n)$ space, they consider more classic streaming with a polylogarithmic number of bits. Here it is not possible to describe the full clustering, so the goal is to estimate the \\emph{cost} of the correlation clustering. They consider $(a,b)$-approximation algorithms where if the optimal cost is $OPT$, the algorithm provides an estimate of at least $OPT$ but at most $a\\cdot OPT+b$. Their first result is a single-pass streaming algorihtm that is an $(O(1),\\delta n^2)$-approximation using $polylog (n)/\\delta^5$ space. Here the constant in the $O$-notation is rather large. However, they also present an algorithm which is an $(3,\\delta n^2)$-approximation in expectation using $2^{O(1/\\delta)} polylog(n)$ space. The first of these results is inspired by the sparse-dense decomposition [AW'22]. At a very high level, using sample based approached, they estimate the number of so called $\\varepsilon$-sparse and $\\varepsilon$-dense edges and relate these to the optimal clustering cost. This can be done for vertices of degree at least $\\delta n$ and the vertices with smaller degrees can contribute at most $\\delta n^2$ to the total cost (this is where the additive term comes from). This is by no means trivial and requires substantial technical work. For the second algorithm, they simulate an algorithm from an unpublished manuscript [BGK13].\n\nThe paper also provides lower bounds showing that an $(1,n^{2-\\varepsilon})$-approximation requires $n^\\varepsilon$ space (this is via a reduction from the INDEX problem) and that an $(1.19,O(n))$-approximation requires $\\Omega(\\sqrt{n})$ space.\n\nThe paper also provides experiments demonstrating the performance of their algorithms on the stochastic block model (SBM) and for Erdos-Renyi random graphs (ER). It is of interest that their algorithms are able to distinguish the clusterable setting in SBM from that the ER $G(n,p)$ which are not clusterable at all.\n\nSTRENGTHS:\nIt is in interesting problem to study the correlation clustering problem in the low space regime both from a theoretical and practical perspective. The paper is well written and the proofs that I looked at in the supplementary material also seem to be clearly written (I only looked at a few of them so I cannot vouch for correctness, however I gave a 4 for soundness based on the ones I checked). The technical contribution of the paper seems impressive, in particular since they also provide lower bounds for the problem.\n\nWEAKNESSES:\nIt is a little unclear which ideas are novel and which ideas are inspired from previous work e.g., [AW'22].\n\nQUESTIONS:\nl85: This seems incorrect. To get sublinear space, you put $\\delta=n^{-0.19}$ and then the additive error is $n^{1.81}$ \n\nl104-110: This section is a little clumsily written and could use a rewriting to make it clearer \n\nl119: What does $0.04\\% \\sim 3.6\\%$ mean? \n\nl276-277: $S_i$ and $m_u$ have not been defined \n\nl347-348: This seems unsurprising. Could it be proved by some union bound approach?\n\nLIMITATIONS:\nThe authors have adequately addressed limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-28"}

comments: SUMMARY:
The paper gives polylog space streaming algorithm for approximately computing the value of the cost of correlation clustering. An algorithm for finding the (approximate) correlation clustering in polylog space is known to be not possible. It is also known that approximately computing the cost within a multiplicative factor is not possible in polylog space. This paper designs polylog space streaming algorithm for approximating the cost within a constant multiplicative plus an O(n^2) additive factor. They show that such an additive factor is necessary. The techniques used are sparse-dense decomposition of [AW22] and local correlation clustering of [BGK13].

STRENGTHS:
1. The theoretical results are novel, well-motivated, and consistent with the known results. All the bounds are well justified. There aren't significant gaps in the presented results.
2. The paper is well written. The results are well-motivated, related work is clearly stated, and the main ideas are clearly outlined in the allowed space.
3. Experimental results show the capability of the algorithm to distinguish between high and low clusterable instances of random graphs generated in the stochastic block model.

WEAKNESSES:
1. Using some space to motivate the practical aspects of cost estimation (instead of computing the solution) will be useful for the readers. In the current version, this aspect of the motivation is pointed to the previous works.
2. The experimental section also does not help with point (1) since it separates random graphs based on clusterability. A discussion on a real world example may help.

QUESTIONS:
- Some of the other queries are mentioned above within the other fields.

LIMITATIONS:
This work is mostly theoretical. There are no negative societal impacts.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors initiate a study of the correlation clustering problem in streams, in the setting when only the *cost* of the clustering needs to be output. Prior work on correlation clustering in streams focused on the \"semi-streaming\" model, in which the clustering must be output but a space of $\\tilde\\Theta(n)$ is allowed. On the other hand, the authors obtain algorithms using space $o(n)$ and even $\\mathrm{poly}\\log(n)$ when only the cost but not the clustering itself is required. The problem of computing the cost is motivated as a measure of the \"clusterability\" cost, and can be used to determine whether it makes sense to compute a clustering or not. \n\nThe authors provide both upper bounds and lower bounds for this problem. For upper bounds, the authors show that in $\\mathrm{poly}\\log(n)$ bits of space, one can obtain a mixed additive-multiplicative error guarantee, with $O(1)$ multiplicative error and $o(n^2)$ additive error. The authors also give show that a purely relative error guarantee cannot be obtained in $\\mathrm{poly}\\log(n)$ space by showing that an approximation with $c = O(1)$ multiplicative error and $\\epsilon n$ additive error requires $\\Omega(\\sqrt n)$ space, for some choices of constant $c$ and $\\epsilon$. Another lower bound that the authors give show that an algorithm achieving a purely additive $n^{2-\\epsilon}$ error requires $\\Omega(n^\\epsilon)$ space. Thus, one cannot hope for too much better than the algorithms given by the authors in general. Furthermore, the authors argue that the $o(n^2)$ additive error is in fact highly useful in practice. Indeed, for a typical application of detecting \"clustered\" communities in a stochastic block model where edges within communities are connected with probability p > 0.5 (say 0.8) and edges between communities with probability p < 0.5 (say 0.2), the $o(n^2)$ additive error algorithm suffices. The practical uses of the algorithm are verified empirically.\n\nIn terms of techniques, the authors give two upper bounds, one based on the classic Pivot algorithm and another based on sparse-dense decompositions developed in prior work. In both cases, the authors carefully implement the algorithms in the streaming setting by running the algorithms only on \"heavy hitter\" vertices and by gathering necessary statistics using sketching techniques.\n\nSTRENGTHS:\nThis work provides both a novel streaming problem with many interesting open questions and nice practical motivations, as well as an interesting and highly nontrivial upper bound. The gap between the upper bounds and lower bounds in this work are highly intriguing, and I would expect this work to stimulate many interesting follow-up works to tighten the results. Thus, the problem that is introduced is extremely interesting. The algorithmic techniques used in the upper bound seem to be a combination of prior work (known algorithms for correlation clustering combined with heavy hitters), but making this go through requires a lot of work in the analysis and is highly nontrivial.\n\nWEAKNESSES:\nJust a couple of comments on the empirical results (however this is not that significant since the contribution of this work is primarily theoretical):\n* The axes in Figure 1 (and all the figures in the appendix in the supplementary material) are illegible. Please take advantage of the space availability in the supplementary material to provide better plots.\n* If $n = 1000, 2000$, then a semi-streaming algorithm would hypothetically use only roughly n/n^2 ~ 0.1% of the edges, so 4% or 10% of the edges does not seem that impressive in comparison at first glance. Are there any comparisons to semi-streaming algorithms in terms of empirical performance?\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nThe authors have thoroughly discussed the limitations (e.g. the additive error) and justified it (e.g. lower bounds).", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the correlation clustering problem in the streaming setting. Unlike previous work they consider algorithms with space much smaller than the number of vertices and only find the cost of the optimal clustering, not the clustering itself. They define an (alpha, beta) approximation to be an additive error of alpha*OPT + beta. Their \"result 1\" and \"result 2\" are for any constant delta > 0 streaming algorithms with approximations of (O(1), delta n^2) and (3, delta n^2) respectively and space O~(1/delta^5) and O~(2^O(1/delta)) respectively. Their \"result 3\" and \"result 4\" are impossibility results. They also give some experimental results.\n\nSTRENGTHS:\nThe paper is well written.\n\nThe impossibility results (\"result 3\" and \"result 4\") look interesting, though I haven't had time to read the proofs.\n\nWEAKNESSES:\nThe first and biggest weakness of this paper is the \"result 1\" and \"result 2\" in the intro can straightforwardly be beaten using known techniques. In particular one can get a (1, delta n^2) approximation in poly(delta) space and time poly(n) + 2^poly(1/delta). To do this simply take a random sample of k = Theta~(1/delta^4) vertices, solve correlation clustering on the sample with an algorithm with delta k^2 additive error, and then scale up the cost by a factor of (n/k)^2. (If one-sided error is desired as in this paper's definitions you also need to subtract roughly delta n^2 from the estimate.) The key idea is the fact that to get delta n^2 additive error it suffices to consider clusterings with O(1/delta) clusters, which makes correlation clustering an instance of the well-studied MAX-CSP family. A sample size of k being enough to approximate the value of MAX CSPs to an additive error first appeared in a paper by Arora, Frieze, K and Karpinski (AFKK) in the 1990s IIRC. (I'll try to track down the exact citation and some more details in a comment to be added this weekend.) For the approximation algorithm one can either use that general-purpose AFKK algorithm or one of the published PTASs for correlation clustering with a constant number of clusters, e.g. [8] in https://en.wikipedia.org/w/index.php?title=Correlation_clustering&oldid=1147372897 (which includes this problem despite the title not mentioning it). The algorithms I mentioned aren't very practical so for experiments you could use applying a branch and bound solver such as SciP, CPLEX, or Gurobi to the correlation clustering integer program for the sample only, which should scale to a sample size of something like 50 vertices, or use any algorithm for correlation clustering (but with worse performance guarantee). To do the random sampling in a streaming fashion keep track of the k vertices with the smallest hashes seen so far and any edges between them.\n\nAnother big weakness of this paper is that correlation clustering instances large enough to not fit on one machine and hence be interesting for streaming algorithms are likely to be sparse, i.e. have o(delta n^2) positive edges. In practice one can therefore probably do better than their result 1 or result 2 (or the algorithm I sketched above) by simply returning the number of positive weight edges, which is a trivial upper-bound on OPT (it's the cost of the all-singletons clustering). I can't rule out the existence of applications where this sort of guarantee would be useful, I'm just not aware of any.\n\nAnother weakness of this paper is streaming models are only useful for a somewhat narrow circumstance: instances are small enough that you have enough time to pass all the data through one machine but large enough that you don't have space to store the full data on one machine. Many streaming results also work in a distributed model such as MPC, which is more practically relevant in my experience, but the authors don't discuss distributed models so I can't tell without reading everything in detail.\n\nQUESTIONS:\nAny comments on the weaknesses mentioned in that section?\n\n=== Misc comments not really questions but I'm not sure where else to put them =====\n\nAbstract: replace \"improves the additive error further down\" with \"further reduces the additive error\"\n\nCorrelation clustering appeared (but by a different name) in a 1989 paper by GROTSCHEL and WAKABAYASI titled \"A CUTTING PLANE ALGORITHM FOR A CLUSTERING PROBLEM\"\n(https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.468.4826&rep=rep1&type=pdf). I'm guessing there are probably even earlier papers somewhere. So the 2004 paper you cite as introducing the correlation clustering problem actually reintroduced it.\n\ncost(C) is defined twice, first in section 2.1 and then in section 2.2. Also the definition in section 2.1 is incomplete since it doesn't say whether the cost is the number of disagreements or the number of agreements or something else, though the word \"cost\" does hint that it's probably the number of disagreements.\n\nUsing the shape of the brackets to distinguish N^-[] from N^-() and N^+[] from N^+() is a bit weird since when bracket shape is used to distinguish notations the two notations are usually totally unrelated, not variations of the same thing. I would prefer a more traditional notation, e.g. overbar on the N for the version that includes the vertex itself.\n\nLIMITATIONS:\nSee weaknesses section for some limitations. No societal impact issues.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}