Memory-Efficient Approximation Algorithms for Max-k-Cut and Correlation Clustering

MAX-K-CUT and correlation clustering are fundamental graph partitioning problems . For a graph G = ( V , E ) with n vertices , the methods with the best approximation guarantees for MAX-K-CUT and the MAX-AGREE variant of correlation clustering involve solving SDPs with O ( n2 ) constraints and variables . Largescale instances of SDPs , thus , present a memory bottleneck . In this paper , we develop simple polynomial-time Gaussian sampling-based algorithms for these two problems that use O ( n + |E| ) memory and nearly achieve the best existing approximation guarantees . For dense graphs arriving in a stream , we eliminate the dependence on |E| in the storage complexity at the cost of a slightly worse approximation ratio by combining our approach with sparsification . 1 Introduction . Semidefinite programs ( SDPs ) arise naturally as a relaxation of a variety of problems such as k-means clustering [ 5 ] , correlation clustering [ 6 ] and MAX-K-CUT [ 14 ] . In each case , the decision variable is an n × n matrix and there are d = Ω ( n2 ) constraints . While reducing the memory bottleneck for large-scale SDPs has been studied quite extensively in literature [ 9 , 11 , 19 , 36 ] , all these methods use memory that scales linearly with the number of constraints and also depends on either the rank of the optimal solution or an approximation parameter . A recent Gaussian-sampling based technique to generate a near-optimal , near-feasible solution to SDPs with smooth objective function involves replacing the decision variable X with a zero-mean random vector whose covariance is X [ 27 ] . This method uses at most O ( n+ d ) memory , independent of the rank of the optimal solution . However , for SDPs with d = Ω ( n2 ) constraints , these algorithms still use Ω ( n2 ) memory and provide no advantage in storage reduction . In this paper , we show how to adapt the Gaussian sampling-based approach of [ 27 ] to generate an approximate solution with provable approximation guarantees to MAX-K-CUT , and to the MAX-AGREE variant of correlation clustering on a graph G = ( V , E ) with arbitrary edge weights using only O ( |V |+ |E| ) memory . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 1.1 MAX-K-CUT . MAX-K-CUT is the problem of partitioning the vertices of a weighted undirected graph G = ( V , E ) into k distinct parts , such that the total weight of the edges across the parts is maximized . If wij is the edge weight corresponding to edge ( i , j ) ∈ E , then the cut value of a partition is CUT = ∑ i and j are in different partitions wij . Consider the standard SDP relaxation of MAX-K-CUT max X 0 〈C , X〉 subject to { diag ( X ) = 1 Xij ≥ − 1k−1 i 6= j , ( k-Cut-SDP ) where C = k−12k LG is a scaled Laplacian . Frieze and Jerrum [ 14 ] developed a randomized rounding scheme that takes an optimal solution X ? of ( k-Cut-SDP ) and produces a random partitioning satisfying E [ CUT ] = ∑ ij∈E , i < j wijPr ( i and j are in different partitions ) ≥ αk〈C , X ? 〉 ≥ αkoptGk , ( 1 ) where optGk is the optimal k-cut value and αk = min−1/ ( k−1 ) ≤ρ≤1 kp ( ρ ) ( k−1 ) ( 1−ρ ) , where p ( ρ ) is the probability that i and j are in different partitions given that Xij = ρ . The rounding scheme proposed in [ 14 ] , referred to as the FJ rounding scheme in the rest of the paper , generates k i.i.d . samples , z1 , . . . , zk ∼ N ( 0 , X ? ) and assigns vertex i to part p , if [ zp ] i ≥ [ zl ] i for all l = 1 , . . . , k . 1.2 Correlation clustering . In correlation clustering , we are given a set of |V | vertices together with the information indicating whether pairs of vertices are similar or dissimilar , modeled by the edges in the sets E+ and E− respectively . The MAX-AGREE variant of correlation clustering seeks to maximize C = ∑ ij∈E− w−ij1 [ i , j in different clusters ] + ∑ ij∈E+ w+ij1 [ i , j in the same cluster ] . Define G+ = ( V , E+ ) and G− = ( V , E− ) . A natural SDP relaxation of MAX-AGREE [ 6 ] is max X 0 〈C , X〉 subject to { diag ( X ) = 1 Xij ≥ 0 i 6= j , ( MA-SDP ) where C = LG− +W+ , LG− is the Laplacian of the graph G− and W+ is the weighted adjacency matrix of the graph G+ . Charikar et al . [ 10 ] ( see also Swamy [ 30 ] ) propose a rounding scheme that takes an optimal solution X ? G of ( MA-SDP ) and produces a random clustering C satisfying E [ C ] ≥ 0.766〈C , X ? G〉 ≥ 0.766optGCC , ( 2 ) where optGCC is the optimal clustering value . The rounding scheme proposed in [ 10 ] , referred to as the CGW rounding scheme in the rest of the paper , generates either k = 2 or k = 3 i.i.d . zero-mean Gaussian samples with covariance X ? G and uses them to define 2 k clusters . 1.3 Contributions . We now summarize key contributions of the paper . Gaussian sampling for MAX-K-CUT . Applying Gaussian sampling-based Frank-Wolfe given in [ 27 ] directly to ( k-Cut-SDP ) uses n2 memory . We , however , show how to extend the approach from [ 27 ] to MAX-K-CUT by proposing an alternate SDP relaxation for the problem , and combining it with the FJ rounding scheme to generate a k-cut with nearly the same approximation guarantees as stated in ( 1 ) ( see Proposition 1 ) using O ( n+ |E| ) memory . A key highlight of our approach is that while the approximation ratio remains close to the state-of-the-art result in ( 1 ) , reducing it by a factor of 1− 5 for ∈ ( 0 , 1/5 ) , the memory used is independent of . We summarize our result as follows . Proposition 1 . For ∈ ( 0 , 1/5 ) , our O ( n2.5|E|1.25 2.5 log ( n/ ) log ( |E| ) ) -time method outlined in Section 3 uses O ( n+ |E| ) memory and generates a k-cut for the graph G = ( V , E ) whose expected value satisfies E [ CUT ] ≥ αk ( 1− 5 ) optGk , where optGk is the optimal k-cut value . Gaussian sampling for MAX-AGREE . The structure of ( MA-SDP ) is similar to ( k-Cut-SDP ) , however , the cost matrix in ( MA-SDP ) is no longer PSD or diagonally dominant , a property that plays an important role in our analysis in the case of MAX-K-CUT . Despite this , we show how to generate a ( 1− 7 ) 0.766-optimal clustering using O ( n+ |E| ) memory . Our approach makes a small sacrifice in the approximation ratio ( as compared to ( 2 ) ) , however , the memory used remains independent of . Proposition 2 . For ∈ ( 0 , 1/7 ) , our O ( n2.5|E|1.25 2.5 log ( n/ ) log ( |E| ) ) -time method outlined in Section 4 uses O ( n+ |E| ) memory and generates a clustering of graph G = ( V , E ) whose expected value satisfies E [ C ] ≥ 0.766 ( 1− 7 ) optGCC , where optGCC is the optimal clustering value . The constructive proof outline of Propositions 1 and 2 is given in Sections 3 and 4 respectively . Memory reduction using graph sparsification . Propositions 1 and 2 state that the memory used by our approach is O ( n + |E| ) . However , for dense graphs , the memory used by our method becomes Θ ( n2 ) . In this setting , to reduce the memory used , we first need to change the way we access the problem instance . We assume that the input ( weighted ) graph G arrives edge-by-edge , eliminating the need to store the entire dense graph . We then replace it with a τ -spectrally close graph G̃ ( see Definition 1 ) with O ( n log n/τ2 ) edges . Next , we compute an approximate solution to the new problem defined on the sparse graph using O ( n log n/τ2 ) memory . For MAX-K-CUT and MAX-AGREE , we show that this method generates a solution with provable approximation guarantees . 1.4 Literature review . We first review key low memory algorithms for linearly constrained SDPs . Burer and Monteiro [ 9 ] proposed a nonlinear programming approach which replaces the PSD decision variable with its low-rank factorization in SDPs with d linear constraints . If the selected value of rank r satisfies r ( r + 1 ) ≥ 2d and the constraint set is a smooth manifold , then any second-order critical point of the nonconvex problem is a global optimum [ 8 ] . Another approach , that requires Θ ( d+ nr ) working memory , is to first determine ( approximately ) the subspace in which the ( low ) rank-r solution to an SDP lies and then solve the problem over the ( low ) r-dimensional subspace [ 11 ] . Alternatively , randomized sketching to a low dimensional subspace is often used as a low-memory alternative to storing a matrix decision variable [ 31 , 34 ] . Recently , such sketched variables have been used to generate a low-rank approximation of a near-optimal solution to SDPs [ 37 ] . The working memory required to compute a near-optimal solution and generate its rank-r approximation using the algorithmic framework proposed by Yurtsever et al . [ 37 ] is O ( d + rn/ζ ) for some sketching parameter ζ ∈ ( 0 , 1 ) . Gaussian sampling-based Frank-Wolfe [ 27 ] usesO ( n+ d ) memory to generate zero-mean Gaussian samples whose covariance represents a near-optimal solution to the SDPs with d linear constraints . This eliminates the dependency on the rank of the near-optimal solution or the accuracy to which its low rank approximation is computed . However , the two problems considered in this paper have SDP relaxations with n2 constraints , for which applying the existing low-memory techniques provide no benefit since the memory requirement of these techniques depends on the number of constraints in the problem . These problems have been studied extensively in literature as we see below . MAX-K-CUT . MAX-K-CUT and its dual MIN-K-PARTITION have applications in frequency allocation [ 12 ] and generating lower bound on co-channel interference in cellular networks [ 7 ] . These problems have been studied extensively in the literature [ 20 , 26 , 29 ] . The SDP-based rounding scheme given in [ 14 ] has also been adapted for similar problems of capacitated MAX-K-CUT [ 16 ] and approximate graph coloring [ 21 ] . In each case , however , the SDP relaxation has Ω ( n2 ) constraints . Alternative heuristic methods have been proposed in [ 13 , 17 , 23 , 25 ] , however , these methods generate a feasible cut which only provides a lower bound on the optimal cut value . Correlation clustering . Charikar et al . [ 10 ] , Swamy [ 30 ] provide 0.766-approximation schemes for MAX-AGREE each of which involve solving ( MA-SDP ) . For large-scale applications , data streaming techniques have been studied quite extensively for various clustering problems , such as k-means and kmedian [ 3 , 24 ] . Ahn et al . [ 1 ] propose a single-pass , Õ ( |E|+n −10 ) -time 0.766 ( 1− ) -approximation algorithm for MAX-AGREE that uses Õ ( n/ 2 ) memory . In contrast , to achieve the same approxima- tion guarantee , our approach usesO ( n+ min { |E| , n logn 2 } ) memory which is equal toO ( n+ |E| ) for sparse graphs , and is independent of . Furthermore , the computational complexity of our approach has a better dependence on given by O ( n2.5 2.5 min { |E| , n logn 2 } 1.25 log ( n/ ) log ( |E| ) ) which is at most O ( n3.75 5 ( log n ) 1.25 log ( n/ ) log ( |E| ) ) . Moreover , our approach is algorithmically simple to implement .

Summary: This paper studies several methods for speeding up / reducing memory of computing max k-way cuts, via methods such as sampling and sparsification. They rigorously prove that the semi-definite program can be approximated to an error of \epsilon in about m \epsilon^{-2} space, and furthermore, the quality of approximations are preserved with the use of spectral graph sparsifiers.  A Matlab implemention of this method is then tested on a multitude of clustering instances, showing that the memory usage is quite low, but at a cost of a fairly large (between 15~40%) relative error.