Title: Single-Pass Pivot Algorithm for Correlation Clustering. Keep it simple!
Venue: NeurIPS 2023
Accepted: yes
Keywords: correlation clustering

Pivot algorithm

streaming
Abstract: We show that a simple single-pass semi-streaming variant of the Pivot algorithm for Correlation Clustering gives a (3+eps)-approximation using O(n/eps) words of memory. This is a slight improvement over the recent results of Cambus, Kuhn, Lindy, Pai, and Uitto, who gave a (3+eps)-approximation using O(n log n) words of memory, and Behnezhad, Charikar, Ma, and Tan, who gave a 5-approximation using O(n) words of memory. One of the main contributions of our paper is that the algorithm and its analysis are simple and easy to understand.

Single-Pass Pivot Algorithm for Correlation Clustering. Keep it simple!

Abstract

We show that a simple single-pass semi-streaming variant of the Pivot algorithm for Correlation Clustering gives a (3 + ε ) -approximation using O ( n/ε ) words of memory. This is a slight improvement over the recent results of Cambus, Kuhn, Lindy, Pai, and Uitto, who gave a (3 + ε ) -approximation using O ( n log n ) words of memory, and Behnezhad, Charikar, Ma, and Tan, who gave a 5-approximation using O ( n ) words of memory. One of the main contributions of this paper is that both the algorithm and its analysis are very simple, and also the algorithm is easy to implement.

1 Introduction

In this paper, we provide a simple single-pass streaming algorithm for Correlation Clustering. The Correlation Clustering model was introduced by Bansal, Blum, and Chawla [2004]. In this model, we have a set of objects represented by vertices of a graph. A noisy binary classifier labels all edges of the graph with '+' and '-' signs. We call edges labeled with a '+' sign - positive edges and edges labeled with a '-' sign - negative edges. A positive edge indicates that the endpoints of the edge are similar, and a negative edge indicates that the endpoints are dissimilar. Given a labeled graph, our goal is to find a clustering that disagrees with the minimum number of edge labels. A positive edge disagrees with the clustering if its endpoints are in different clusters, and a negative edge disagrees with the clustering if its endpoints are in the same cluster. We assume that the classifier can make many errors and, consequently, there may be no clustering that agrees with all labels of the graph.

Correlation Clustering has been widely studied in ML and theoretical computer science communities. It was used for image-segmentation (Kim et al. [2014]), link prediction (Yaroslavtsev and Vadapalli [2018]), document clustering (Bansal et al. [2004]), community detection (Veldt et al. [2018], Shi et al. [2021]), and other problems.

In Section 1.1, we discuss various objectives and assumptions used for Correlation Clustering. We now focus on the most common setting, where the graph G is a complete graph (i.e., we have a '+' or '-' label for each pair of vertices u and v ), the classifier can make arbitrary mistakes, and the objective is to minimize the number of disagreements.

In their original paper, Bansal, Blum, and Chawla [2004] gave a constant factor approximation for Correlation Clustering on complete graphs. Charikar, Guruswami, and Wirth [2005] improved the approximation factor to 4. Then, Ailon, Charikar, and Newman [2008] proposed combinatorial and LP-based PIVOT algorithms, with approximation factors 3 and 2 . 5 , respectively. The LP-based PIVOT algorithm was further improved by Chawla, Makarychev, Schramm, and Yaroslavtsev [2015], who provided a 2 . 06 approximation. Finally, Cohen-Addad, Lee, and Newman [2022b] and Cohen-Addad, Lee, Li, and Newman [2023] used a Sherali-Adams relaxation for the problem to obtain 1 . 994 + ε and 1 . 73 + ε approximations, respectively. On the hardness side, Charikar, Guruswami, and Wirth [2005] proved that the problem is APX-Hard and thus unlikely to admit a PTAS.

Most abovementioned algorithms use linear programming (LP) for finding a clustering. The number of constraints in the standard LP is Θ( n 3 ) (due to 'triangle inequality constraints'). Solving such LPs is prohibitively expensive for large data sets 2 . That is why, in practice, the algorithm of choice is the combinatorial PIVOT algorithm by Ailon et al. [2008] along with its variants. The approximation factor of this algorithm is 3.

To deal with massive data sets, researchers started exploring parallel (MPC) and semi-streaming algorithms. Recent works on MPC algorithms include Blelloch, Fineman, and Shun [2012], Chierichetti, Dalvi, and Kumar [2014], Pan, Papailiopoulos, Oymak, Recht, Ramchandran, and Jordan [2015], Ahn, Cormode, Guha, McGregor, and Wirth [2015], Ghaffari, Gouleakis, Konrad, Mitrovi´ c, and Rubinfeld [2018], Fischer and Noever [2019], Cambus, Choo, Miikonen, and Uitto [2021], CohenAddad, Lattanzi, Mitrovi´ c, Norouzi-Fard, Parotsidis, and Tarnawski [2021], Assadi and Wang [2022], Cambus, Kuhn, Lindy, Pai, and Uitto [2022], Behnezhad, Charikar, Ma, and Tan [2022].

Streaming algorithms have been extensively studied by Chierichetti, Dalvi, and Kumar [2014], Ahn, Cormode, Guha, McGregor, and Wirth [2015], Ghaffari, Gouleakis, Konrad, Mitrovi´ c, and Rubinfeld [2018], Cohen-Addad, Lattanzi, Mitrovi´ c, Norouzi-Fard, Parotsidis, and Tarnawski [2021], Assadi and Wang [2022], Cambus, Kuhn, Lindy, Pai, and Uitto [2022], Behnezhad, Charikar, Ma, and Tan [2022, 2023]. This year, Behnezhad, Charikar, Ma, and Tan [2023] designed a single-pass polynomial-time semi-streaming 5 -approximation algorithm that uses O ( n ) words of memory. Then, Cambus, Kuhn, Lindy, Pai, and Uitto [2022] provided a single-pass polynomial-time semi-streaming ( 3 + ε )-approximation algorithm that uses O ( n log n ) words of memory. Both algorithms are based on the combinatorial PIVOT algorithm.

Our Results. In this paper, we examine a simple semi-streaming variant of the Ailon, Charikar, and Newman [2008] combinatorial PIVOT algorithm that uses O ( n/ε ) words of memory and gives a 3 + ε approximation. Our algorithm needs less memory than the algorithm by Cambus, Kuhn, Lindy, Pai, and Uitto [2022] and gives the same (3 + ε ) approximation guarantee. The algorithm is very simple and easy to implement. Its analysis is simple and concise.

We note that one can take advantage of the low memory footprint of our algorithm even in a nonstreaming setting. We provide some experimental results in Appendix (see Figure 2).

Semi-streaming Model. We consider the following single-pass semi-streaming model formally defined by Feigenbaum, Kannan, McGregor, Suri, and Zhang [2005] (see also Muthukrishnan [2005]). The algorithm gets edges of the graph along with their labels from an input stream. The order of edges in the graph is arbitrary and can be adversarial. The algorithm can read the stream only once. We assume that the algorithm has O ( kn ) words of memory, where k is a constant. Thus, it cannot store all edges in the main memory. As in many previous papers on streaming algorithms for Correlation Clustering, we shall assume that the stream contains only positive edges (if negative edges are present in the stream, our algorithm will simply ignore them). We prove the following theorem.

Theorem 1.1. Algorithm 1 (see Figure 1) is a randomized polynomial-time semi-streaming algorithm with an approximation factor 3 + O (1 /k ) . The algorithm uses O ( kn ) words of memory, where each word can store numbers between 1 and n . The algorithm spends at most O (log k ) units of time for every edge it reads in the streaming phase. Then, it requires additional O ( kn ) units of time to find a clustering (in the pivot selection and clustering phase).

1.1 Related Work

In this paper, we consider the most commonly studied objective, MINDISAGREE, of minimizing the number of disagreements. Swamy [2004] and Charikar, Guruswami, and Wirth [2005] investigated a complementary objective, MAXAGREE, of maximizing the number of agreements. They gave 0 . 766 -approximation algorithms for MAXAGREE. Puleo and Milenkovic [2016], Chierichetti, Kumar, Lattanzi, and Vassilvitskii [2017], Charikar, Gupta, and Schwartz [2017], Ahmadi, Khuller, and Saha [2019], Kalhan, Makarychev, and Zhou [2019], Ahmadian, Epasto, Kumar, and Mahdian [2020], Friggstad and Mousavi [2021], Jafarov, Kalhan, Makarychev, and Makarychev [2021], Schwartz and Zats [2022], Ahmadian and Negahbani [2023], Davies, Moseley, and Newman [2023] considered several variants of fair and local objectives.

Initialization phase

Pick a random ordering of vertices π : V →{ 1 , . . . , n } .

For each vertex u ∈ V :

-Create a priority queue A ( u ) capped at size k and initialize A ( u ) = { u } .

Streaming phase

For each edge ( u, v ) ∈ E :

-Add u to A ( v ) . Add v to A ( u ) . Remove the lowest ranked elements from A ( u ) and A ( v ) , if necessary, to make sure that they contain at most k elements.

Pivot selection and clustering

For each not-yet clustered vertex u ∈ V chosen in the order of π :

-Find the highest ranked vertex v in A ( u ) such that v = u or v is a pivot i.e.,

-If such vertex v exists, place u in the cluster of v . If u = v , mark u as a pivot .

-Else, place u in a singleton cluster. Mark u as a special singleton .

For each pivot vertex u , output the cluster consisting of all vertices clustered with u . Then, output all special singleton vertices, each in its own clusters.

Charikar, Guruswami, and Wirth [2005] and Demaine, Emanuel, Fiat, and Immorlica [2006] provided O (log n ) approximation for Correlation Clustering on incomplete graphs and edge-weighed graphs. For the case when all edge weights are in the range [ α, 1] , Jafarov, Kalhan, Makarychev, and Makarychev [2020] gave a (3 + 2 ln 1 / α ) -approximation algorithm.

The problem has also been studied under the assumption that the classifier makes random or semirandom errors by Bansal, Blum, and Chawla [2004], Elsner and Schudy [2009], Mathieu and Schudy [2010], Makarychev, Makarychev, and Vijayaraghavan [2015]. Online variants of Correlation Clustering were considered by Mathieu, Sankur, and Schudy [2010], Lattanzi, Moseley, Vassilvitskii, Wang, and Zhou [2021], Cohen-Addad, Lattanzi, Maggiori, and Parotsidis [2022a].

2 Single-Pass Streaming Algorithm

Our algorithm is an extension of the algorithm by Behnezhad, Charikar, Ma, and Tan [2023]. Loosely speaking, the algorithm works as follows: It picks a random ranking of vertices, scans the input stream and keeps only the k top-ranked neighbours for every vertex u . Then, it runs the Ailon, Charikar, and Newman [2008] PIVOT algorithm on the graph that contains only edges from every vertex to its k top-ranked neighbours. We provide the details below.

We shall assume that every vertex is connected with itself by a positive edge. In other words, we include each vertex in the set of its own (positive) neighbours. We denote the set of positive neighbours of u by N ( u ) . The algorithm first picks a random ordering of vertices π : V →{ 1 , . . . , n } . We say that vertex u is ranked higher than vertex v according to π if π ( u ) < π ( v ) . Therefore, π -1 (1) is the highest ranked vertex, and π -1 ( n ) is the lowest ranked vertex. For every vertex u , we will keep track of its k highest-ranked neighbours. We denote this set by A ( u ) . We initialize each A ( u ) = { u } . Then, for every positive edge ( u, v ) in the input stream, we add u to the set of v 's neighbours A ( v ) and v to the set of u 's neighbours A ( u ) . If A ( u ) or A ( v ) get larger than k , we remove the lowest ranked vertices from these sets to make sure that the sizes of A ( u ) and A ( v ) are at most k . After we finish reading all edges from the input stream, we run the following variant of the PIVOT algorithm. We mark all vertices as non-pivots. Then, we consider vertices according to their ranking (i.e., π -1 (1) , π -1 (2) , . . . , π -1 ( n ) ). For every vertex u , we find the highest ranked neighbour v in A ( u ) , such that v = u or v is a pivot . If such v exists, we place u in the cluster of v . Moreover, if v = u , we declare u a pivot . If no such v in A ( u ) exists, we put u in its own cluster and declare it a special singleton . We provide pseudo-code for this algorithm in Figure 1.

We use fixed-size priority queues to store sets A ( u ) . Whenever we need to add a vertex v to A ( u ) , we insert v into A ( u ) and remove the lowest-ranked element from A ( u ) if the size of A ( u ) is greater

than k . We remark that even though each u is always added to set A ( u ) in the beginning of the algorithm, it may later be removed if u is not among its k top-neighbours.

We can reduce the memory usage of the algorithm by adding u to A ( v ) only if π ( u ) < π ( v ) and adding v to A ( u ) only if π ( v ) < π ( u ) . This change does not affect the resulting clustering because u is always added to A ( u ) , and, consequently, we never pick a pivot v with π ( v ) > π ( u ) at the clustering phase of the algorithm. Nevertheless, to simplify the exposition, we will assume that we always add u to A ( v ) and v to A ( u ) .

In this paper, we will call clusters special singleton clusters if they were created in the 'else' clause of the clustering phase. That is, if they contain a single vertex u , which is not a pivot. Note that some other clusters may contain a single vertex as well. We will not call them special singleton clusters.

Before analyzing the approximation factor of the algorithm, we briefly discuss its memory usage and running time. For every vertex u , we keep its rank π ( u ) , a set of its k highest ranked neighbors A ( u ) , a pointer to the pivot if u is not a special singleton, and two bits indicating whether u is a pivot and whether u is a special singleton cluster. Thus, the total memory usage is O ( kn ) words; each word is an integer that can store a number between 1 and n .

The running time of the initialization step is O ( n ) . The worst-case running time of the streaming phase is O ( m log k ) where m is the number of positive edges (the algorithm needs to perform a constant number of priority queue operations for each edge in the stream; the cost of each operation is O (log k ) ). The expected running time of the streaming phase of the algorithm is 3 O ( m + nk log k log(2 m/ ( nk ))) if m>nk . It is O ( m ) if m>nk log k log log k . The running time of the clustering phase is O ( kn ) , since we need to examine all neighbors v of u in queue A ( u ) .

2.1 Approximation Factor

We now show that the approximation factor of the algorithm is 3 + O (1 /k ) . For the sake of analysis, we consider an equivalent variant of our algorithm that reveals the ordering π one step at a time. This variant of the algorithm is not a streaming algorithm, per se. However, it produces exactly the same clustering as our single-pass streaming algorithm. Thus, it is sufficient to show that it has a 3 + O (1 /k ) approximation. The algorithm, which we will refer to as Algorithm 2 , works as follows. Initially, it marks all vertices as unprocessed and not clustered. It also sets a counter K 1 ( u ) = 0 for each u . We denote the set of all unprocessed vertices at step t by V t and the set of all non-clustered vertices by U t . Then, V 1 = V and U 1 = V . At step t , the algorithm picks a random vertex w t in V t . If w t is not yet clustered ( w t ∈ U t ), then w t is marked as a pivot, a new cluster is created for w t , and all not yet clustered neighbours of w t including w t itself are added to the new cluster. If w t is already clustered, then the counter K t ( v ) is incremented by 1 for all not yet clustered neighbours v of w t (i.e., K t +1 ( v ) = K t ( v ) + 1 ). After that, all vertices v , whose counter K t +1 ( v ) got equal to k at this step, are put into special singleton clusters. We mark w t as processed (by letting V t +1 = V t \ { w t } ) and mark all vertices clustered at this step as clustered (by removing them from U t ).

We show that this algorithm outputs the same clustering as Algorithm 1. To this end, define an ordering π for Algorithm 2 as π : w t ↦→ t , where w t is the vertex considered at step t of Algorithm 2. Clearly, π is a random (uniformly distributed) permutation. We prove the following lemma.

Lemma 2.1. If Algorithm 1 and Algorithm 2 use the same ordering π , then they produce exactly the same clustering of V .

Proof. We show that every vertex u is assigned to the same pivot v in both algorithms, or u is put in a special singleton cluster by both algorithms assuming that Algorithms 1 and 2 use the same ordering π . The proof is by induction on the rank π ( u ) . Consider a vertex u and assume that all vertices v higher ranked than u are clustered in the same way by Algorithms 1 and 2. Both algorithms examine

neighbours of u in exactly the same order, π . Moreover, they only consider the top k neighbours: Algorithm 1 does so because it stores neighbours of u in a priority queue of size k ; Algorithm 2 does so because it keeps track of the number, K t ( u ) , of examined neighbours of u and once that number equals k , it puts u in a special singleton cluster. Both algorithms assign u to the first found pivot vertex v or to a special singleton cluster if no such v exists among the top k neighbours of u . Hence, both algorithms cluster u in the same way.

We now analyze Algorithm 2. We split all steps of Algorithm 2 into pivot and singleton steps. We say that step t is a pivot step if vertex w t which was processed at this step was not clustered at the beginning of the step (i.e., w t ∈ U t ). We say that t is a singleton step, otherwise (i.e., w t ∈ V t \ U t ). Observe that all clusters created at pivot steps are not special singleton clusters (even though some of them may contain a single vertex; see above for the definition of special singleton clusters). In contrast, all clusters created at singleton steps are special singleton clusters. We will now show that the expected cost of clusters created at pivot steps is at most 3 OPT . Then, we will analyze singleton steps and prove that the cost of clusters created at those steps is at most O (1 /k ) OPT , in expectation (where OPT is the cost of the optimal solution).

We say that a positive or negative edge ( u, v ) is settled at step t if both u and v were not clustered at the beginning of step t but u , v , or both u and v were clustered at step t . Note that once an edge ( u, v ) is settled, we can determine if ( u, v ) is in agreement or disagreement with the clustering produced by the algorithm: For a positive edge ( u, v ) ∈ E + if both u and v belong to the newly formed cluster, then ( u, v ) is in agreement with the clustering. For a negative edge ( u, v ) ∈ E -if both u and v belong to the newly formed cluster, then ( u, v ) is in disagreement with the clustering. Similarly, if only one of the endpoints u or v belongs to the new cluster, then ( u, v ) is in agreement with the clustering for a negative edge ( u, v ) ∈ E -and in disagreement with the clustering for a positive edge ( u, v ) ∈ E + . In the latter case, we say that a positive edge ( u, v ) ∈ E + is cut by the algorithm, and we call ( u, v ) a cut edge .

We say that the cost of a settled edge is 0 if it agrees with the clustering and 1 if it disagrees with the clustering. Let P be the cost of all edges settled during pivot steps. Furthermore, let P + be the cost of all positive edges and P -be the cost of all negative edges settled during pivot steps. Then, P = P + + P -. If v is a singleton cluster, let S ( v ) be the number of positive edges incident to v the algorithm cut when it created this cluster. Note, that some edges incident to v might have been cut before v was put in a singleton cluster. We do not count these edges in S ( v ) .

Every edge which is in disagreement with the clustering is (1) a positive edge cut at a pivot step of the algorithm; (2) a negative edge joined at a pivot step of the algorithm; or (3) a positive edge cut at a singleton step of the algorithm. Hence, the cost of the algorithm equals:

Lemma 2.2. We have E [ P + + P -] ≤ 3 OPT .

Remark: This lemma implies that E [ P + ] ≤ 3 OPT . We note that we could get a slightly stronger bound: E [ P + ] ≤ 2 OPT , which would yield a slightly stronger upper bound on E [ ∑ v S ( v )] (see below).

Proof. Observe that pivot steps are identical to the steps of the PIVOT algorithm. That is, given that w t belongs to U t , the conditional distribution of w t is uniform in U t and the cluster created at this step contains w t and all its positive neighbours. Let us fix an optimal solution. Let O ∗ be the set of edges in the optimal solution that disagree with the classifier. Denote the cost of the optimal solution by OPT = | O ∗ | . Ailon, Charikar, and Newman [2008] proved that the expected cost of the solution produced by the PIVOT algorithm is upper bounded by 3 OPT . Furthermore, they showed that the expected cost of edges settled at step t is upper bounded by 3 E [∆ OPT t ] , where ∆ OPT t is the number of edges from O ∗ settled by PIVOT at step t (see also Lemma 4 in Chawla et al. [2015]). Therefore, the expected total cost of all edges settled at pivot steps of our algorithm is upper bounded by 3 E [ ∑ t is pivot step ∆ OPT t ] . The number of all edges in O ∗ settled at pivot steps is upper bounded by the size of O ∗ , which equals OPT . Hence, E [ P + + P -] ≤ 3 OPT .

We now bound E [ ∑ v S ( v )] . Fix a vertex v . Let N ( v ) be the set of all vertices connected with v by a positive edge ('positive or similar neighbours'), and | N ( v ) | be the number of positive neighbours of

v including v itself. We now define a new quantity X t ( v ) . If v is not in a singleton cluster at step t , then let X t ( v ) be the number of positive edges incident on v cut in the first t -1 steps of the algorithm. Otherwise, let X t ( v ) be the number of positive edges incident on v cut by the algorithm before v was placed in the singleton cluster. The number of edges incident to v cut by the algorithm equals

X n +1 ( v ) + S ( v ) . Hence, the total number of cut positive edges equals 1 2 ∑ v ∈ V ( X n +1 ( v ) + S ( v ) ) . It also equals P + + ∑ v ∈ V S ( v ) . Thus, ∑ v ∈ V X n +1 ( v ) = 2 P + + ∑ v ∈ V S ( v ) and

We now show that E [ S ( v )] ≤ 1 / k E [ X n +1 ( v )] for all v and, therefore,

and

To finish the analysis of Algorithm 2, it remains to prove the following lemma.

Lemma 2.3. We have E [ S ( v )] ≤ 1 / k E [ X n +1 ( v )] .

Proof. Define the following potential function:

where 1 ( v / ∈ U t ) is the indicator of event { v / ∈ U t } . Let F t be the state of the algorithm at the beginning of step t . We claim that Φ t ( u ) is a submartingale i.e., E [Φ t +1 ( u ) | F t ] ≥ Φ t ( u ) .

Claim 2.4. Φ t ( v ) is a submartingale.

Proof. Consider step t of the algorithm. If v is already clustered (i.e., v / ∈ U t ), then all edges incident on v are settled and terms 1 ( v / ∈ U t ) , K t ( v ) , and X t ( v ) no longer change. Hence, the potential function Φ t ( v ) does not change as well.

Suppose that v ∈ U t . If at step t , the algorithm picks w t not from the neighbourhood N ( v ) , then v does not get clustered at this step and also K t ( v ) does not change. The value of X t ( v ) may increase as some neighbours of v may get clustered. However, X t ( v ) cannot decrease. Therefore, Φ t +1 ( v ) ≥ Φ t ( v ) , and E [ Φ t +1 ( v ) -Φ t ( v ) | w t / ∈ N ( v ); F t ] ≥ 0 .

Let us now assume that at step t , the algorithm chooses w t from the neighbourhood of v . Then, v is clustered with conditional probability at least

because if the pivot is chosen in N ( v ) ∩ U t , then v is put in the cluster of w t . We have | N ( v ) ∩ U t | = | N ( v ) | -X t ( v ) , since every clustered neighbour of v contributes 1 to the number of cut edges X t ( v ) , and the total number of v 's positive neighbours equals | N ( v ) | . Also, note that | N ( v ) ∩ V t | ≤ | N ( v ) | . Hence,

With this conditional probability, 1 ( v / ∈ U t ) gets equal to 1 at step t , and, as a result, Φ t ( v ) increases by X t +1 ( v ) ≥ X t ( v ) . With the remaining probability, K t ( v ) increases by 1 , and, as a result, Φ t ( v ) decreases by at most | N ( v ) | -X t +1 ( v ) ≤ | N ( v ) | -X t ( v ) . Thus,

This concludes the proof.

Since Φ 1 ( v ) = 0 and Φ t ( v ) is a submartingale, we have E [Φ n +1 ( v )] ≥ 0 . Therefore,

If v is in a singleton cluster, then K n +1 ( v ) = k and X n +1 ( v ) + S ( v ) = | N ( v ) | (because all positive edges incident to v are cut). Hence,

If v is not in a singleton cluster, then S ( v ) = 0 . Thus, we always have kS ( v ) ≤ K n +1 ( v ) · ( | N ( v ) | -X n +1 ( v )) . Consequently, E [ X n +1 ( v )] ≥ k E [ S ( v )] .

Acknowledgments and Disclosure of Funding

Konstantin Makarychev is supported by NSF Awards CCF-1955351, CCF-1934931, EECS-29 2216970. The authors would like to thank the anonymous reviewers for their valuable comments.

References

Ahmadi, S. Khuller, and B. Saha. Min-max correlation clustering via multicut. In Integer Programming and Combinatorial Optimization: 20th International Conference, IPCO 2019, Ann Arbor, MI, USA, May 22-24, 2019, Proceedings , pages 13-26. Springer, 2019.

Ahmadian and M. Negahbani. Improved approximation for fair correlation clustering. In International Conference on Artificial Intelligence and Statistics , pages 9499-9516. PMLR, 2023.

Ahmadian, A. Epasto, R. Kumar, and M. Mahdian. Fair correlation clustering. In International Conference on Artificial Intelligence and Statistics , pages 4195-4205. PMLR, 2020.

Ahn, G. Cormode, S. Guha, A. McGregor, and A. Wirth. Correlation clustering in data streams. In International Conference on Machine Learning , pages 2237-2246. PMLR, 2015.

Ailon, M. Charikar, and A. Newman. Aggregating inconsistent information: ranking and clustering. Journal of the ACM (JACM) , 55(5):1-27, 2008.

Assadi and C. Wang. Sublinear time and space algorithms for correlation clustering via sparsedense decompositions. In 13th Innovations in Theoretical Computer Science Conference (ITCS 2022) , 2022.

Bansal, A. Blum, and S. Chawla. Correlation clustering. Machine learning , 56:89-113, 2004.

Behnezhad, M. Charikar, W. Ma, and L.-Y. Tan. Almost 3-approximate correlation clustering in constant rounds. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) , pages 720-731. IEEE, 2022.

Behnezhad, M. Charikar, W. Ma, and L.-Y. Tan. Single-pass streaming algorithms for correlation clustering. In Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 819-849. SIAM, 2023.

E. Blelloch, J. T. Fineman, and J. Shun. Greedy sequential maximal independent set and matching are parallel on average. In Proceedings of the twenty-fourth annual ACM symposium on Parallelism in algorithms and architectures , pages 308-317, 2012.

Cambus, D. Choo, H. Miikonen, and J. Uitto. Massively parallel correlation clustering in bounded arboricity graphs. In 35th International Symposium on Distributed Computing (DISC 2021) . Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2021.

Cambus, F. Kuhn, E. Lindy, S. Pai, and J. Uitto. A parallel algorithm for (3 + ε ) -approximate correlation clustering. arXiv preprint arXiv:2205.07593 , 2022.

Charikar, V. Guruswami, and A. Wirth. Clustering with qualitative information. Journal of Computer and System Sciences , 71(3):360-383, 2005.

Charikar, N. Gupta, and R. Schwartz. Local guarantees in graph cuts and clustering. In Integer Programming and Combinatorial Optimization: 19th International Conference, IPCO 2017, Waterloo, ON, Canada, June 26-28, 2017, Proceedings , pages 136-147. Springer, 2017.

Chawla, K. Makarychev, T. Schramm, and G. Yaroslavtsev. Near optimal lp rounding algorithm for correlationclustering on complete and complete k-partite graphs. In Proceedings of the fortyseventh annual ACM symposium on Theory of computing , pages 219-228, 2015.

Chierichetti, N. Dalvi, and R. Kumar. Correlation clustering in mapreduce. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 641-650, 2014.

Chierichetti, R. Kumar, S. Lattanzi, and S. Vassilvitskii. Fair clustering through fairlets. Advances in neural information processing systems , 30, 2017.

Cohen-Addad, S. Lattanzi, S. Mitrovi´ c, A. Norouzi-Fard, N. Parotsidis, and J. Tarnawski. Correlation clustering in constant many parallel rounds. In International Conference on Machine Learning , pages 2069-2078. PMLR, 2021.

Cohen-Addad, S. Lattanzi, A. Maggiori, and N. Parotsidis. Online and consistent correlation clustering. In International Conference on Machine Learning , pages 4157-4179. PMLR, 2022a.

Cohen-Addad, E. Lee, and A. Newman. Correlation clustering with sherali-adams. In 2022 IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) , pages 651-661. IEEE, 2022b.

Cohen-Addad, E. Lee, S. Li, and A. Newman. Handling correlated rounding error via preclustering: A 1.73-approximation for correlation clustering. In 2023 IEEE 64th Annual Symposium on Foundations of Computer Science (FOCS) . IEEE, 2023.

Davies, B. Moseley, and H. Newman. Fast combinatorial algorithms for min max correlation clustering. In 40th International Conference on Machine Learning (ICML'23) , 2023.

D. Demaine, D. Emanuel, A. Fiat, and N. Immorlica. Correlation clustering in general weighted graphs. Theoretical Computer Science , 361(2-3):172-187, 2006.

Elsner and W. Schudy. Bounding and comparing methods for correlation clustering beyond ilp. In Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing , pages 19-27, 2009.

Feigenbaum, S. Kannan, A. McGregor, S. Suri, and J. Zhang. On graph problems in a semistreaming model. Theoretical Computer Science , 348(2-3):207-216, 2005.

Fischer and A. Noever. Tight analysis of parallel randomized greedy mis. ACM Transactions on Algorithms (TALG) , 16(1):1-13, 2019.

Friggstad and R. Mousavi. Fair correlation clustering with global and local guarantees. In Algorithms and Data Structures: 17th International Symposium, WADS 2021, Virtual Event, August 9-11, 2021, Proceedings 17 , pages 414-427. Springer, 2021.

Ghaffari, T. Gouleakis, C. Konrad, S. Mitrovi´ c, and R. Rubinfeld. Improved massively parallel computation algorithms for mis, matching, and vertex cover. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing , pages 129-138, 2018.

Jafarov, S. Kalhan, K. Makarychev, and Y. Makarychev. Correlation clustering with asymmetric classification errors. In International Conference on Machine Learning , pages 4641-4650. PMLR, 2020.

Jafarov, S. Kalhan, K. Makarychev, and Y. Makarychev. Local correlation clustering with asymmetric classification errors. In International Conference on Machine Learning , pages 4677-4686. PMLR, 2021.

Kalhan, K. Makarychev, and T. Zhou. Correlation clustering with local objectives. Advances in Neural Information Processing Systems , 32, 2019.

Kim, C. D. Yoo, S. Nowozin, and P. Kohli. Image segmentation usinghigher-order correlation clustering. IEEE transactions on pattern analysis and machine intelligence , 36(9):1761-1774, 2014.

Lattanzi, B. Moseley, S. Vassilvitskii, Y . Wang, and R. Zhou. Robust online correlation clustering. Advances in Neural Information Processing Systems , 34:4688-4698, 2021.

Makarychev, Y. Makarychev, and A. Vijayaraghavan. Correlation clustering with noisy partial information. In Conference on Learning Theory , pages 1321-1342. PMLR, 2015.

Mathieu and W. Schudy. Correlation clustering with noisy input. In Proceedings of the twenty-first annual ACM-SIAM symposium on Discrete Algorithms , pages 712-728. SIAM, 2010.

Mathieu, O. Sankur, and W. Schudy. Online Correlation Clustering. In J.-Y. Marion and T. Schwentick, editors, 27th International Symposium on Theoretical Aspects of Computer Science , volume 5 of Leibniz International Proceedings in Informatics (LIPIcs) , pages 573-584, Dagstuhl, Germany, 2010. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik. ISBN 978-3939897-16-3. doi: 10.4230/LIPIcs.STACS.2010.2486. URL http://drops.dagstuhl.de/ opus/volltexte/2010/2486 .

Muthukrishnan. Data streams: Algorithms and applications. Foundations and Trends in Theoretical Computer Science , 1(2):117-236, 2005.

Pan, D. Papailiopoulos, S. Oymak, B. Recht, K. Ramchandran, and M. I. Jordan. Parallel correlation clustering on big graphs. Advances in Neural Information Processing Systems , 28, 2015.

Puleo and O. Milenkovic. Correlation clustering and biclustering with locally bounded errors. In International Conference on Machine Learning , pages 869-877. PMLR, 2016.

Schwartz and R. Zats. Fair correlation clustering in general graphs. In Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (APPROX/RANDOM 2022) . Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2022.

Shi, L. Dhulipala, D. Eisenstat, J. Ł˘ acki, and V. Mirrokni. Scalable community detection via parallel correlation clustering. Proceedings of the VLDB Endowment , 14(11):2305-2313, 2021.

Swamy. Correlation clustering: maximizing agreements via semidefinite programming. In SODA , volume 4, pages 526-527. Citeseer, 2004.

Veldt. Correlation clustering via strong triadic closure labeling: Fast approximation algorithms and practical lower bounds. In International Conference on Machine Learning , pages 22060-22083. PMLR, 2022.

Veldt, D. F. Gleich, and A. Wirth. A correlation clustering framework for community detection. In Proceedings of the 2018 World Wide Web Conference , pages 439-448, 2018.

Yaroslavtsev and A. Vadapalli. Massively parallel algorithms and hardness for single-linkage clustering under l p distances. In 35th International Conference on Machine Learning (ICML'18) , 2018.

A Figures

1

2

3

4

5

6

7

8

9

10

0

.

95

1

1

.

05

1

.

1

1

.

15

1

.

2

1

.

25

1

.

3

1

.

35

1

.

4

1

.

45

1

.

5

cost

k

K

-nn graph for Gaussian mixture with

K

= 25

; 100,000 points

K

-nn graph for Gaussian mixture with

K

= 50

; 200,000 points

Peer reviews:
comments: SUMMARY:
The paper shows a single-pass semi streaming algorithm for correlation clustering (minimizing disagreements) which computes a  $3+\epsilon$ approximate solution using $O(n/\epsilon)$ words. This should be compared with:
 * a 5-approximate algorithm using O(n) words
 * a $3+\epsilon$ approximate algorithm using $O(n \log n)$ words

The algorithm is a variant of the PIVOT algorithm combined with a simple sparsification technique which keeps $1/\epsilon$ edges per each node.

STRENGTHS:
* The paper improves the space usage for an important problem
* The presentation is clear. Actually the entire paper, including all proofs (excl references) fits in 6 pages

WEAKNESSES:
While I appreciate the theoretical result and the simplicity, the improvement over the previous results seems quite small: i.e. it can be seen as either slightly reducing the approximation factor or space per vertex from $O(\log n)$ to $O(1/\epsilon)$. I'm not convinced that the problem of correlation clustering in a streaming setting is popular enough for this kind of improvement to be of sufficient interest to NeurIPS audience

QUESTIONS:
I'm afraid I can't think of a question that could affect my recommendation

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-06-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the correlation clustering problem in the semi-streaming model obtaining a $(3+\\varepsilon)$-approximation to the problem with $O(\\frac{n}{\\varepsilon})$ words of memory. This is an improvement over [CKLPU'22] who gave the same approximation with $O(n\\log n)$ words of memory (I'm not sure of the dependence on $\\varepsilon$) and over [BCMT'22] who gave a 5-approximation with $O(n)$ words of memory. Moreover, the algorithm is extremely simple. It picks a a random ranking of the vertices and for each vertex keeps a list of its $k=O(1/\\varepsilon)$ highest ranked neighbors during the streaming phase. In post processing the algorithm goes through the vertices according to their rank, for each vertex $u$ finding the highest ranked vertex $v$ in its list such that $v=u$ or $v$ is a pivot. In the former case, $u$ is declared a pivot, in the latter case, $u$ is assigned to the cluster of $v$. If no such $v$ exists, $u$ is put in a singleton cluster. The analysis describes an equivalent offline algorithm which is analysed partly via an analogue to the  Pivot algorithm from [ACN'08] and partly via a martingale argument for singleton vertices relating their cost to that of the Pivot algorithm.\n\nSTRENGTHS:\nThe paper is very well written and the proofs are neat and readily checked to be correct. The algorithm is simple and I think it is nice and rare with a short paper with such a clean idea. Since furthermore, correlation clustering is of interest to many people in the Neurips community I think the paper should be accepted.\n\nWEAKNESSES:\nPerhaps the authors could have considered running some experiments on the quality of the clustering output by their algorihtm.\n\nQUESTIONS:\nWhat's the dependence on $\\varepsilon$ in [CKLPU'22]?\n\nl66: \".... has $O(n)$ words of memory where $k$ is a constant\": Do you mean $O(nk)$?\n\nl131: I think the runtime bound can be improved to $O((k\\log k \\log n)n+m)$ or something like that. Indeed, with the random order, you only need to update the priority queue with probability $k/i$ when the $i$'th neighbor arrives. \n\nl142: You define $U_t$ to be the unclustered vertices but latter on in the martingale argument (l206, l209, l216,...) you refer to $U_t$ as the set of clustered vertices. I think the indicator in the potential should be $1(v\\notin U_t)$.\n\nl195-196: Can you define $\\Delta OPT_t$ more clearly. When you say the optimal cost of the edges settled by PIVOT, you mean the cost of these edges in the optimal solution, right?\n\nl212 \"...must be a pivot chosen among non-clustered...\": Couldn't it also be because it's counter increased to $k$? It doesn't seem to matter for the argument?\n\nEquation above l217: The first inequality is an equality, right? And $v\\in U_{n+1}$ should be  $v\\notin U_{n+1}$?\n\nLIMITATIONS:\nNo limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper demonstrates that the celebrated Pivot algorithm for (unweighted, complete) correlation clustering (with the min disagreements objective) can be adapted to the single-pass, semi-streaming setting, to yield a (3+\\eps)-approximation in expectation using O(n/eps) words of memory. (Note that the original Pivot algorithm gives a 3-approximation in expectation.) This improves on previous works that show a (3+\\eps)-approximation using O(n\\log n) words of memory, and a 5-approximation using O(n) words of memory. In the single-pass, semi-streaming setting, edges and their labels arrive online adversarially and the stream can only be read once. It is assumed that only positive edges arrive and the remaining edges in the complete graph are implicitly negative. \n\nThe Pivot algorithm chooses a random permutation of the vertices, visits each vertex in this order, and if a vertex is unclustered, makes that vertex a \"pivot.\" The pivot grabs its unclustered positive neighbors from the remaining sequence and forms a cluster. The algorithm proposed in this paper tweaks the Pivot algorithm by again choosing a random permutation of the vertices, but for each vertex it only retains positive edges to the top-k ranked neighbors in this permutation. It then runs Pivot on this subgraph, using the chosen random permutation. The analysis rests on controlling the cost of (positive) edges cut by special vertices called \"singleton vertices.\" These are vertices u whose top-k positive neighbors all come before u, but none are chosen as pivots, so u ends up in a singleton cluster. (Controlling the cost of other edges is done by reusing the analysis of Pivot, since these edges are cut by the pivot vertices.) The crux of the argument controlling the cost of singleton vertices is to show that a certain potential function is always positive in expectation, which in turn comes from showing it is a submartingale.\n\nSTRENGTHS:\n- The algorithm is a pleasingly simple variant of the Pivot algorithm. It lends insight into just how much \"wiggle room\" there is in the original Pivot algorithm, as it might a priori seem surprising that we only need retain the top-k ranked neighbors of each vertex. \n\n- The analysis (particularly the main lemma, which designs a novel potential function and shows it is a submartingale) is quite elegant. Moreover, it is substantially simpler than analyses in the aforementioned previous works on the semi-streaming setting. \n\n- This paper contributes to the growing literature on how the Pivot algorithm can be adapted to other models of computation (in addition to streaming, parallel models such as MapReduce). While the result itself is not a significant improvement over known results, and the algorithm is of comparable simplicity to previous ones, it provides a much cleaner analysis.\n\nWEAKNESSES:\nWhile the paper is overall clear and well-written, it would be useful to give a bit more background comparing the proposed algorithm to those in prior work in the semi-streaming setting.  \n\nSome typos / omissions in the writing: \n- U_t on pg. 6 is seemingly meant as the complement of how it is defined on pg. 4 \n- D^+(v) = |N(v)| on pg. 5 \n- It would be good to add a line stating formally what it means to \"cut\" at an edge (although it is clear from context)\n\nAnd a suggestion: The use of the term singleton is a bit confusing, since, as the authors mention, not all singleton clusters correspond to \"singletons\" as defined in the Pivot selection and clustering section of Figure 1. It may be worth just using a distinct term altogether for such vertices.\n\nQUESTIONS:\nn/a\n\nLIMITATIONS:\nn/a", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper provides a streaming algorithm for correlation clustering on complete graphs that uses O(n/eps) space and has an approximation factor of 3+eps. The algorithm is based on the combinatorial pivot algorithm. 

The prior results in the streaming setting give 5 approx in the better space of O(n); and another result with 3+eps approx in space O(n log n). They are also based on the combinatorial Pivot algorithm.

The best offline approximation for the problem achieves a factor below 2, but the best approximation factor based on the combinatorial pivot algorithm achieves a factor of 3.

STRENGTHS:
Matches the best offline approximation based on combinatorial Pivot algorithm

The algorithm is simple and thus practical to implement, and the proof is very clean.

WEAKNESSES:
They provide a code in the supplementary material but no experimental results comparing this algorithm with the prior works.

QUESTIONS:
Minor comments:
Page 1, sings -> signs
In description of Algorithm 2, did you mean U_1 = V as opposed to U_1=U?

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-17

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of correlation clustering on a complete graph, where all the edges are assigned either \"+\" or \"-\". The goal is to come up with a clustering that maximizes agreement with the edge labels. This is a well studied problems in approximation algorithms. There is an elegant 3 approximation due to ACN called PIVOT: pick unclustered vertices in random order, create a cluster for each such vertex and all unclustered neighbors that have a + edge to it. The analysis is similarly simple and elegant. \n\nThis paper considers the semi-streaming model. where the edges arrive one at a time. It is not obvious how the ACN version of PIVOT works in this model, since we do not know all the neighbors of a vertex v when processing it. A recent paper due to Behnezhad et al gives a 5 approximation for a variant of PIVOT. Their algorithm  gives a 5 approximation in O(n) space, but its analysis is subtle and a little tedious. \n\nThis paper presents another variant of pivot which gives a 3 +eps approximation, again in O(n/eps) space. The analysis is arguably simpler (but to my taste, not as simple as the original PIVOT algorithm).  Previously, there was a 3 +eps algorithm, but that needed n log n space.\n\nSTRENGTHS:\nIt is nice to have both a simple algorithm and a simple analysis for a very natural problem. This paper deserves to be published somewhere.\n\nWEAKNESSES:\nThe contribution feels somewhat incremental.  We had a simple algorithm that gave a 5 approximation in O(n) space, as well as a 3 + eps approximation in O(n log n) space. This paper gives 3 + eps with a simple algorithm and analysis, and O(n/eps) space. In some sense, it gets the best of all worlds. \n\nI am not entirely convinced that either then space reduction or factor improvement ranks as a important enough contribution at a top ML venue like neurips. For calibration, I dont think this would get into a top tier theory venue, but I'd be fine with accepting it to a next-tier conference. If some area expert thinks this is the case, I'd be open to changing my mind.\n\nQUESTIONS:\nWhen you say that the stream only consists of positive edges, it is worht pointing out that you are working on the complete graph and the remainder of the edges are assumed to be negative.\n\nLIMITATIONS:\nnone", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-26"}