Title: Learning-Augmented Approximation Algorithms for Maximum Cut and Related Problems
Venue: NeurIPS 2024
Accepted: yes
Keywords: learning-augmented algorithm

approximation algorithm

maximization

CSPs

learning with advice
Abstract: In recent years, there has been a surge of interest in the use of machine-learned predictions to bypass worst-case lower bounds for classical problems in combinatorial optimization. So far, the focus has mostly been on online algorithms, where information-theoretic barriers are overcome using predictions about the unknown future. In this paper, we consider the complementary question of using learned information to overcome computational barriers in the form of approximation hardness of polynomial-time algorithms for NP-hard (offline) problems. We show that noisy predictions about the optimal solution can be used to break classical hardness results for maximization problems such as the max-cut problem and more generally, maximization versions of constraint satisfaction problems (CSPs).

Learning-Augmented Approximation Algorithms for Maximum Cut and Related Problems

Abstract

In recent years, there has been a surge of interest in the use of machine-learned predictions to bypass worst-case lower bounds for classical problems in combinatorial optimization. So far, the focus has mostly been on online algorithms, where information-theoretic barriers are overcome using predictions about the unknown future. In this paper, we consider the complementary question of using learned information to overcome computational barriers in the form of approximation hardness of polynomial-time algorithms for NP-hard (offline) problems. We show that noisy predictions about the optimal solution can be used to break classical hardness results for maximization problems such as the max-cut problem and more generally, maximization versions of constraint satisfaction problems (CSPs).

1 Introduction

The design and analysis of algorithms beyond the classical worst-case paradigm has been an active area of research (see, e.g., the collection of surveys by Roughgarden [2020]). In recent years, this has been accelerated by the success and widespread adoption of machine learning models across application domains, leading researchers to ask: can we use machine-learned information to solve typical instances of a problem better than what we can hope for in the worst case? This meta-question has been particularly influential in the realm of online optimization, where the goal is to design algorithms for inputs that are revealed sequentially over time. Indeed, assuming that the future unfolds in a typical rather than worst-case manner, we can compensate for the information deficit of the online algorithm with predictions learned based on past data, thereby helping it bypass information-theoretic lower bounds. This principle has been successfully applied to a broad range of online problems, such as caching [Lykouris and Vassilvitskii, 2021], rent-or-buy [Purohit et al., 2018], covering Bamas et al. [2020b], network design Azar et al. [2022], scheduling Azar et al. [2021], matching Dinitz et al. [2021], and many others (see related work for more references).

In this paper, we study the role of machine-learned predictions in offline NP-hard problems. For offline problems, an algorithm has no information disadvantage compared to an optimal solution: the disadvantage is computational. The NP-hardness of problems makes exact algorithms that run in polynomial time unlikely. This has led to the field of approximation algorithms, where the goal is to obtain polynomial-time algorithms that output solutions that are approximately optimal. In

particular, an approximation algorithm for an optimization problem has an approximation factor of α if the solution it produces on every instance is within a factor of α of that of an optimal solution. The best approximation factor for an NP-hard offline problem is also subject to computational barriers. For instance, for the classical MAXCUT problem, it is known that the approximation factor of α GW ≈ 0 . 878 obtained in the celebrated work of Goemans and Williamson [1995] is the best possible under the Unique Games Conjecture (UGC). This raises a natural question: can we use machine-learned predictions to overcome computational barriers to approximation algorithms for NP-hard problems?

1.1 Our Contributions

Suppose we are given a noisy prediction that is mildly correlated with an optimal solution for a given problem instance. Can we use such a prediction to recover a better approximation to the optimal solution than is possible without any prediction? For (strongly) NP-hard problems, we typically know barriers for the best approximation factor α that can be achieved by polynomial-time algorithms. Using a machine-learned prediction of an optimal solution, we seek to go beyond this barrier: to obtain a polynomial-time algorithm with an approximation factor strictly better than α . The quantum of improvement naturally depends on the quality of the prediction: if the prediction is ε -correlated with the target solution, can we get an α + f ( ε ) approximation?

To make these questions concrete, we first consider the MAXCUT problem in the learning-augmented setting. Given an undirected, weighted graph, the MAXCUT problem asks for a bi-partition of the vertices such that the total weight of edges in the cut is maximized. Assuming the widely-accepted unique games conjecture, the Goemans-Williamson approximation bound of α GW ≈ 0 . 878 is the best possible for a polynomial-time algorithm. But, suppose we are given a prediction for the optimal max-cut that is independently correct for every vertex with probability 1 / 2 + ε , for any ε > 0 . (Note that random guessing achieves correctness of 1 / 2 ; so, we assume that the prediction is only ε -better than random guesses.) Can this prediction be used to obtain an approximation factor better than α GW in polynomial time? This question was posed by Svensson in his SODA 2023 plenary lecture.

Our first result is to unconditionally improve on worst-case performance for the MAXCUT problem using an ε -correlated prediction. This answers the question posed by Svensson in the affirmative. Specifically, we give an algorithm that obtains an ( α GW + ˜ Ω( ε 4 )) -approximate MAXCUT solution. This also quantifies the dependence between the improvement in the approximation factor and the correlation of the prediction with the actual optimal solution. Interestingly, we also relax the independence requirement to just pairwise independence of the predictions on the vertices. This is significant because in practice, the predictions for the different vertices are likely to be obtained from a machine learning model or a human expert, either of which sources are unlikely to output completely independent predictions for different vertices.

We further complement this result by considering another natural prediction model where instead of a noisy prediction for every vertex, we get a correct prediction but only for an ε -fraction of randomly chosen vertices. (To distinguish between these models, we call the former noisy predictions and the latter partial predictions .) In this case, we obtain an ( α RT + Ω( ε )) -approximate solution to MAXCUT, where α RT ≃ 0 . 858 is the approximation factor obtained by Raghavendra and Tan for the MAXBISECTION problem [Raghavendra and Tan, 2012]. Note that α RT is slightly smaller than α GW, but we get a better advantage of Ω( ε ) instead of Ω( ε 4 ) .

Next, we show that our algorithmic framework is applicable beyond the MAXCUT problem. Constraint Satisfaction Problems (CSPs) are a broad class of optimization problems that includes fundamental optimization tasks such as MAXk -SAT, MAXk -LIN, MAXk -AND, etc. In particular, 2 -CSPs are a subclass of CSPs where each constraint contains only two variables. This includes problems such as MAXCUT, MAXDICUT, and MAX-2-SAT. A classical result of Arora et al. [1999] showed that it is possible to obtain an approximation factor arbitrarily close to 1 for 'dense' instances of all 2-CSPs including MAXCUT. We show that ε -correlated predictions of the optimal solution is a useful tool for dense instances of all 2 -CSPs. In particular, we use the prediction to lower the 'density threshold' for obtaining an arbitrarily small approximation factor for dense instances of 2-CSPs. In other words, the assumption on the density of the instance (which is a function of the prediction bias and the approximation error) for our result in the learning-augmented setting is weaker than that of Arora et al. [1999], i.e., our result applies to a broader set of instances.

1.2 Related Work

In recent years, the abundance of data and the tremendous success of machine learning has led to a variety of attempts at going beyond traditional worst-case analysis for combinatorial optimization by taking advantage of learned information. In clustering, Ashtiani et al. [2016] introduced a setting where an algorithm can query an external oracle (e.g., a machine learning model) to learn if a pair of points are in the same cluster in the optimal clustering (same-cluster queries). The goal then is to recover an optimal solution (or a sufficiently good approximation) while minimizing the number of oracle queries. Tight bounds have been obtained for various clustering objectives in this setting, from k -means [Ailon et al., 2018] to correlation clustering [Mazumdar and Saha, 2017]. Moreover, robust settings that incorporate noise in the oracle answers have also been studied Larsen et al. [2020], Del Pia et al. [2022]. Another line of work Ergun et al. [2022], Gamlath et al. [2022] considers k -means and related clustering problems where the algorithm is provided noisy node labels. For instance, Gamlath et al. [2022] showed that even when the labels provided by the oracle are correct with a tiny probability (say 1% ), it is possible to obtain a (1 + o (1)) -approximation to the k -means objective as long as the clusters are not too small.

A different line of work has aimed to incorporate machine-learned predictions in the design of online algorithms (see, e.g., the surveys Mitzenmacher and Vassilvitskii [2020, 2022]). This model was introduced by Lykouris and Vassilvitskii for the caching problem Lykouris and Vassilvitskii [2021] and has since been studied in many problem domains such as rent or buy Purohit et al. [2018], Gollapudi and Panigrahi [2019], Anand et al. [2020], covering Bamas et al. [2020b], Anand et al. [2022], Gupta et al. [2022], scheduling Purohit et al. [2018], Wei and Zhang [2020], Bamas et al. [2020a], Lattanzi et al. [2020], Mitzenmacher [2020], Azar et al. [2021], Cohen and Panigrahi [2023], Lindermayr et al. [2023], Lassota et al. [2023], caching Lykouris and Vassilvitskii [2021], Wei [2020], Jiang et al. [2022b], Bansal et al. [2022], matching Dinitz et al. [2021], Lavastida et al. [2021], secretary problems Antoniadis et al. [2020b], Dütting et al. [2021], graph problems Antoniadis et al. [2020a], Jiang et al. [2022a], Almanza et al. [2021], Antoniadis et al. [2023], Bernardini et al. [2022], Anand et al. [2022], Fotakis et al. [2021], Azar et al. [2022], and many others. (The reader is referred to ALP [2023] for a compendium of papers in this area.) The main goal in this line of work is to overcome information-theoretic lower bounds for online problems using machine-learned predictions about the unknown future.

Two recent works (concurrent and independent of ours) have considered the computational complexity of MAXCUT and CSPs in the context of noisy predictions. The first of these is Bampis et al. [2024], who tackle the problem of speeding up approximation schemes for dense CSP instances using noisy predictions. Namely, they provide an algorithm that achieves a (1 -ε ) approximation whose running time depends on the density of the instance and the error in the prediction. The algorithm runs in polynomial time if the number of edges in the instance is Ω( n 2 / log n ) , assuming the prediction label of each vertex is correct with constant probability. The second work is by Ghoshal et al. [2024], who consider both the partial and the noisy predictions models under full independence of the predictions. They provide a (1 -O (( ε √ ∆) -1 )) -approximation for both models, where ∆ is the average degree of the graph. Alternatively, they show that if the edges are weighted, the value of the solution computed is at least opt -√ n ∑ ij w 2 ij ε -1 .

2 Preliminaries

The MAXCUT Problem. We start by describing the MAXCUT problem. In this problem, we are given a weighted graph G = ( V, E ) represented by a (symmetric) n × n adjacency matrix A , where A ij = w ij , the weight of edge { i, j } if it exists, and 0 otherwise. (We assume the graph has no self-loops, and hence A has zeroes on the diagonal.) We let W i = ∑ j w ij denote the weighted degree of vertex i . We use D := diag ( W 1 , . . . , W n ) to denote the diagonal matrix with these weighted degrees, and L = D -A to denote the (unnormalized) Laplacian matrix of the graph. Note that x ∈ {-1 , 1 } n denotes a cut in the graph, and the quadratic form ⟨ x, Lx ⟩ = ∑ { i,j }∈ E w ij ( x i -x j ) 2 counts (four times) the weight of edges crossing the cut between the vertices labeled 1 , and those labeled -1 .

The MAXCUT problem asks for the cut with maximum edge weight. Hence, MAXCUT can be rephrased as follows:

We defer the formal definition of CSPs to Section 5.

The Noisy/Partial Predictions Framework. We describe the prediction models for the MAXCUT problem. We extend the noisy predictions model to general Max2 -CSPs in Section 5.

In the noisy predictions model for MAXCUT, we assume there is some fixed and unknown optimal solution x ∗ ∈ {-1 , 1 } n . The algorithm has access to a prediction vector Y ∈ {-1 , 1 } n , such that for each vertex i , Y i is the correct label x ∗ i with some (unknown) probability 1 / 2 + ε , and is the other (incorrect) label with probability 1 / 2 -ε . Here we only assume pairwise independence; for any two vertices i and j , Pr[ i, j both give their correct labels ] = ( 1 / 2 + ε ) 2 .

In the partial predictions model for MAXCUT, the algorithm has access to a prediction vector Y ∈ {-1 , 0 , 1 } n such that for each vertex i , Y i is the correct label x ∗ i with some (unknown) probability ε and is 0 otherwise. Again, we only assume pairwise independence; for any two vertices i and j , Pr[ i, j both give their correct labels ] = ε 2 .

3 MAXCUT in the Noisy Prediction Model

Recall that in the noisy prediction model, the predicted label of each vertex is its correct label in a fixed max-cut with probability 1 / 2 + ε . Moreover, the labels are pairwise independent. One can show that in this model, if one were to simply output the prediction itself, then its value would be at least O ( m/ 2 + ε 2 (opt -m/ 2)) for an unweighted graphs with m edges. But, this is generally worse than the α GW · opt ≈ 0 . 878 · opt bound obtained by the Goemans-Williamson MAXCUT algorithm. Our main result is to give an algorithm that uses a noisy prediction to outperform α GW in the approximation bound by an additive poly( ε ) factor:

Theorem 3.1 (Noisy Predictions) . Given noisy predictions with a bias of ε , there is a polynomial-time randomized algorithm that obtains an approximation factor of α GW + ˜ Ω( ε 4 ) in expectation for the MAXCUT problem.

The rest of this section is devoted to proving the above theorem. A basic distinction that we will use throughout this section is that of ∆ -wide and ∆ -narrow graphs; these should be thought of as weighted analogs of high-degree and low-degree graphs. We first define these and related concepts below, then we present an algorithm for the MAXCUT problem on ∆ -wide graphs in §3.1, followed by the result for ∆ -narrow graphs in §3.2. We finally wrap up with the proof of Theorem 3.1.

We partition the edges incident to vertex i into two sets: the ∆ -prefix for i comprises the ∆ heaviest edges incident to i (breaking ties arbitrarily), while the remaining edges make up the ∆ -suffix for i . We fix a parameter η ∈ (0 , 1 / 2 ) . We will eventually set ∆ = Θ(1 /ε 2 ) and η to be an absolute constant. Recall that W i = ∑ j ∈ [ n ] A ij is the weighted degree of i .

Definition 3.2 ( ∆ -Narrow/Wide Vertex) . A vertex i is ∆ -wide if the total weight of edges in its ∆ -prefix is at most ηW i , and so the weight of edges in its ∆ -suffix is at least (1 -η ) W i . Otherwise, the vertex i is ∆ -narrow .

Intuitively, a ∆ -wide vertex is one where most of its weighted degree is preserved even if we ignore the ∆ heaviest edges incident to the vertex.

We partition the vertices V = [ n ] into the ∆ -wide and ∆ -narrow sets; these are respectively denoted V > ∆ and V < ∆ . We define W > ∆ := ∑ i ∈ V > ∆ W i and W < ∆ := ∑ i ∈ V < ∆ W i , and hence the sum of weighted degrees of all vertices is W := ∑ n i =1 W i = W > ∆ + W < ∆ .

Definition 3.3 ( ∆ -Narrow/Wide Graph) . A graph is ∆ -wide if the sum of weighted degrees of ∆ -wide vertices accounts for at least 1 -η fraction of that of all vertices; i.e., if W > ∆ ≥ (1 -η ) W . Otherwise, it is ∆ -narrow.

3.1 Solving MAXCUT for ∆ -wide graphs

For ∆ -wide graphs, we show:

Theorem 3.4. Fix ε ′ ∈ (0 , 1) . Given noisy predictions with bias ε , there is a polynomial-time randomized algorithm that, given any ∆ -wide graph, outputs a cut of value at least the maximum cut minus (5 η +2 ε ′ ) W , where ∆ := O (1 / ( ε · ε ′ ) 2 ) , with probability 0 . 98 .

Since the graph is ∆ -wide, most vertices have their weight spread over a large number of their neighbors. In this case, the prediction vector allows us to obtain a good estimate ˆ r of the optimal neighborhood imbalance r ∗ (the difference between the number of neighbors a vertex has on its side versus the other side of the optimal cut). We can then write an LP to assign fractional labels to vertices that maximize the cut value while remaining faithful to these estimates ˆ r ; finally rounding the LP gives the solution.

3.1.1 The ∆ -wide Algorithm

Define an n × n matrix ˜ A from the adjacency matrix A as follows: for each row corresponding to the edges incident to a vertex i , we set the entry ˜ A ij = 0 if the edge ( i, j ) is in the ∆ -prefix of vertex i ; otherwise, ˜ A ij = A ij . Now, define an n -dimensional vector ˆ r as follows:

where Y is the prediction vector. Solve the linear program:

Let ˆ x ∈ [ -1 , 1] n be the optimal LP solution.

Finally, do the following O ( 1 / η ) times independently, and output the best cut X ∗ among them: randomly round the fractional solution ˆ x independently for each vertex to get a cut X ∈ {-1 , 1 } n ; namely, Pr[ X i = 1] = (1+ˆ x i ) / 2 and Pr[ X i = -1] = (1 -ˆ x i ) / 2 .

3.1.2 The Analysis

For a labeling x ∈ {-1 , 1 } n , the neighborhood imbalance for vertex i is defined as ∑ j A ij x j = ( Ax ) i . This denotes the (signed) difference between the total weight of edges incident to i that appear and do not appear in the cut defined by the labeling x . The maximality of the optimal cut x ∗ ∈ {-1 , 1 } n ensures that x ∗ i · sign(( Ax ∗ ) i ) ≤ 0 for all i ; else, switching x i from 1 to -1 or vice-versa increases the objective. Define r ∗ := Ax ∗ as the vector of imbalances for the optimal cut.

Lemma 3.5. The vector ˆ r satisfies

Proof. Observe that

First, we consider a ∆ -narrow vertex i . Since ˆ r i = 0 , we have | ˆ r i -r ∗ i | = | r ∗ i | ≤ W i . So summing over all ∆ -narrow vertices gives

since the graph is ∆ -wide.

Now, we consider a ∆ -wide vertex i . We have

To bound the first term in the RHS of (3), recall that r ∗ i = E [( AZ ) i ] . Thus,

Since E [ Z ] = x ∗ ∈ {-1 , 1 } n , we get

where in the last step, we used the fact that i is a ∆ -wide vertex.

Now, we bound the second term in the RHS of (3). Using Chebyshev's inequality on the sum ( ˜ AZ ) i = ∑ j ˜ A ij Z j , we get

Since the variables Z j are pairwise independent, the variance var (( ˜ AZ ) i ) = ∑ j ˜ A 2 ij var ( Z j ) . The variance of each Z j is O (1 /ε 2 ) . For ∑ j ˜ A 2 ij , we know

Note that the weight of any edge in the ∆ -suffix of i is at most W i / ∆ . Therefore, by our definition of ˜ A , we have ∥ ˜ A i ∥ ∞ ≤ W i / ∆ . Since ˜ A ij ≤ A ij for all j ∈ [ n ] , we also have ∥ ˜ A i ∥ 1 ≤ ∥ A i ∥ 1 = W i . Applying these bounds, we get: ∑ j ∈ [ n ] ˜ A 2 ij ≤ W 2 i / ∆ . Therefore,

Summing over all ∆ -wide vertices, we get

Combining with (2) for ∆ -narrow vertices, we get

Now using Markov's inequality with Lemma 3.5, we get that setting ∆ = Ω(1 / ( εε ′ ) 2 ) for any fixed constant ε ′ > 0 ensures that we get a vector of empirical imbalances ˆ r satisfying

with probability at least 0 . 99 . (Since the 2 ηW losses are deterministically bounded, we can use Markov's inequality only on the random variable ∑ i ∈ V > ∆ | ( ˜ AZ ) i -E [( ˜ AZ ) i ] | .) Hence, when the event in (4) happens, the vector x ∗ is a feasible solution to LP (1).

Next, we need to analyze the quality of the cut produced by randomly rounding the solution of LP (1). Recall that for the (unnormalized) Laplacian L and some x ∈ {-1 , 1 } n , the cut value is

Lemma 3.6. For any ∆ -wide graph, the algorithm from §3.1.1 outputs X ∗ ∈ {-1 , 1 } n that satisfies

with probability at least 0 . 98 .

Proof. Recall that the cut X ∗ is the best among T := O ( 1 / η ) independent roundings of cut ˆ x . Consider one of the roundings X , and write:

Let us first bound the expectation of each of the terms in the RHS of (6) separately.

To bound the first term ⟨ ˆ x, ˆ r ⟩ , note that given (4) (which happens with probability 0 . 99 ), the solution x ∗ is feasible for the LP in (1). This means the optimal solution ˆ x has objective function value

Next, we bound the second term ( ⟨ ˆ x, A ˆ x ⟩ - ⟨ ˆ x, ˆ r ⟩ ) by

by feasibility of ˆ x for the LP in (1). Finally, we bound the third term ( ⟨ X,AX ⟩ - ⟨ ˆ x, A ˆ x ⟩ ) , this time in expectation:

Chaining eqs. (7) to (9) for the various parts of (6), we get

Moreover, using that ⟨ X,AX ⟩ ∈ [ -W,W ] , we get

If X ∗ is the cut with the smallest value of ⟨ X,AX ⟩ among all the independent roundings:

Substituting into the definition of f ( · ) completes the proof.

This proves Lemma 3.6, and hence also Theorem 3.4.

Deterministic Rounding. We observe that we can replace the repetition by a simple pipeage rounding algorithm to round the fractional solution ˆ x to an integer solution X ∗ without suffering any additional loss. Indeed, viewing ⟨ x, Ax ⟩ as a function of some x i keeping the remaining { x 1 , . . . , x n } \ { x i } fixed gives us a linear function of x i (since the diagonals of A are zero). Hence we can increase or decrease the value of x i to decrease ⟨ x, Ax ⟩ until x i ∈ {-1 , 1 } . Iterating over the variables gives the result. However, this does not change the result qualitatively.

3.2 Solving MAXCUT for ∆ -narrow graphs

Next, we consider ∆ -narrow graphs. We show:

Theorem 3.7. For any ∆ ∈ N , there is a randomized algorithm for the MAXCUT problem with an (expected) approximation factor of α GW + ˜ Ω( η 5 / ∆ 2 ) on any ∆ -narrow graph.

For the case of ∆ -narrow graphs, we do not use predictions; rather, we adapt an existing algorithm for the MAXCUT problem for low-degree graphs by Feige et al. [2002] and its refinement due to Hsieh and Kothari [2022]. Note that any graph with maximum degree ∆ is clearly ∆ -narrow (even when η = 1 ).

3.2.1 The ∆ -narrow Algorithm

We show that Theorem 3.7 holds for the Feige, Karpinski, and Langberg (FKL) MAXCUT algorithm [Feige et al., 2002]. We briefly recall this algorithm first. Consider the MAXCUT SDP with triangle inequalities:

where S n is the unit sphere of n dimensions. Let ˆ v be an optimal solution to this SDP.

Let g be a random vector where each coordinate is sampled independently from a standard normal distribution. We use random hyperplane rounding from the MAXCUT algorithm of Goemans and Williamson [1995] to round ˆ v to ˆ x ∈ {-1 , 1 } n as follows: if ⟨ ˆ v i , g ⟩ > 0 , then ˆ x i = 1 ; else, ˆ x i = -1 .

̸

Now, define F = { i ∈ [ n ] : ⟨ ˆ v i , g ⟩ ∈ [ -δ, δ ] } for some δ = Θ(1 / ((∆ /η ) √ log(∆ /η ))) . We partition N i := [ n ] \{ i } as follows: B i := { j ∈ N i \ F : ˆ x j = ˆ x i } , and C i := { j ∈ N i \ F : ˆ x j = ˆ x i } and D i := N i ∩ F . We define F ′ ⊆ F as follows: i ∈ F ′ if i ∈ F and w ( B i ) > w ( C i ) + w ( D i ) where w ( S ) := ∑ j ∈ S A ij . In the final output X ∈ {-1 , 1 } n , we flip the vertices in F ′ , namely X i = -ˆ x i if i ∈ F ′ , else X i = ˆ x i .

We now give an analysis for the FKL algorithm establishing Theorem 3.7.

The 'local gain' for a vertex i ∈ F is defined as ∆ i := ( | B i | -( | D i | + | C i | )) + , where z + = max( z, 0) . We now state the following key lemmas:

Lemma 3.8. For any vertex i ∈ [ n ] , Pr[ i ∈ F ] = Ω( δ ) .

Proof. This lemma immediately follows from [Hsieh and Kothari, 2022, Fact 3].

Let i be a ∆ -narrow vertex, and w ∈ R n be its weight vector ( w i = A ij for all j ∈ [ n ] ) so that W i = ∥ w i ∥ 1 . Let w ′ ∈ R n be the projection of w onto its top ∆ coordinates. The narrowness of i implies that ∥ w ′ ∥ 1 ≥ η ∥ w ∥ 1 , which implies that

It turns out that the analysis of Hsieh and Kothari [2022] still holds under the above bound between ℓ 1 and ℓ 2 norms of weight vectors. So we have the following slight generalization of their Lemma 8. Lemma 3.9 (extends Lemma 8 of Hsieh and Kothari [2022]) . There is a large enough constant C such that for any d ≥ 3 and δ = 1 Cd √ log d , for any vertex i whose weight vector w satisfies ∥ w ∥ 2 ≤ d ∥ w ∥ 2 , it holds that the expected local gain of a vertex i satisfies:

Proof. In Hsieh and Kothari [2022], the only place where the degree bound d is used is ∥ w ∥ 2 1 ≤ d ∥ w ∥ 2 2 at the end of the proof of Lemma 7.

Proof of Theorem 3.7. Note that the value of the cut X exceeds that of ˆ x by ∑ i ∈ F ′ ∆ i , i.e.,

Let the approximation factor of the cut ˆ x output by the Goemans-Williamson algorithm be denoted α GW and let opt be the size of the maximum cut. Then,

From Lemmas 3.8 and 3.9 with d = ∆ /η 2 , we get E [ ⟨ X,LX ⟩ ] ≥ α GW · opt + Ω ( 1 (∆ /η 2 ) √ log(∆ /η 2 ) · ∑ i :∆ -narrow W i (∆ /η 2 ) √ log(∆ /η 2 ) ) . Since ∑ i :∆ -narrow W i ≥ ηW ≥ 2 η · opt , we get E [ ⟨ X,LX ⟩ ] ≥ ( α GW + ˜ Ω( η 5 / ∆ 2 )) · opt .

3.3 Wrapping up: Proof of Theorem 3.1

For ∆ -wide graphs, Theorem 3.4 returns a cut with value at least opt -(2 η + ε ′ ) W with probability 0 . 98 . Since we can always find a cut of value α GW · opt , and opt ≥ W/ 2 , this means the expected cut value is at least [ 0 . 98 · (1 -6 η -2 ε ′ ) + 0 . 02 · α GW ] opt . And for ∆ -narrow graphs, Theorem 3.7 finds a cut with expected value [ α GW + ˜ Ω( η 5 / ∆ 2 ) ] · opt . Moreover, recall that ∆ = O (1 / ( εε ′ ) 2 ) . Setting η, ε ′ to be suitably small universal constants gives us that both the above approximation factors are at least α GW + ˜ Ω( ε 4 ) , which proves Theorem 3.1.

1 2

4 MaxCut in the Partial Prediction Model

We now consider the partial prediction model, where each vertex pairwise-independently reveals their correct label with probability ε . Intuitively, this prediction model provides more information than the noisy prediction model since all predictions are guaranteed to be correct. Indeed, this is reflected in out first bound: we show that since an ε 2 fraction of the edges are induced by the vertices with the given labels, it is easy to get an approximation ratio of almost α GW +Ω( ε 2 ) . (We give details in Appendix A.)

Theorem 4.1. Given noisy predictions with a rate of ε , there is a polynomial-time randomized algorithm that obtains an (expected) approximation factor of α GW + ε 2 for the MAXCUT problem

Although the Ω( ε 2 ) advantage in this theorem is already better than ˜ Ω( ε 4 ) in Theorem 3.1, we ask if can we do even better given the more informative predictions. Ideally, we could get an Ω( ε ) -advantage if the hyperplane rounding performs better than α GW for the edges with only one endpoint's label revealed. One naive way to achieve this is to hope that the rounding preserves the marginals ; i.e., E [ x i ] = ⟨ v 0 , v i ⟩ for all i ∈ [ n ] . In that case, if we consider ( i, j ) where if v i = ± v 0 , the probability that ( i, j ) is cut is exactly their contribution to the SDP (1 -⟨ v i , v j ⟩ ) .

Since the hyperplane rounding does not satisfy this property, we use the rounding scheme developed by Raghavendra and Tan [2012] for max-bisection that has an approximation ratio α RT ≈ 0 . 858 while preserving the marginals. The proof of this theorem is deferred to Appendix A.

Theorem 4.2 (Partial Predictions) . Given partial predictions with a rate of ε , there is a polynomialtime randomized algorithm that obtains an (expected) approximation factor of α RT +(1 -α RT -o (1))(2 ε -ε 2 ) for the MAXCUT problem.

5 2-CSPs in the Noisy Prediction Model

In this section, we go beyond the MAXCUT problem and consider general 2 -CSPs. In particular, we extend Theorem 3.4 to this broader class of problems. Let us first define Max2 -CSPs (Constraint Satisfaction Problems). Each constraint is a predicate on two variables: e.g., AND, OR, Not-Equals, or Xor. We are given a collection of such constraints (each on two variables), and the goal is to find an binary assignment to the variables that satisfy the maximum number of constraints. (E.g., if the predicate is Not-Equals, and constraints form the edges of some graph, we get the MAXCUT problem.)

Formally, for a multi-index α ∈ [ n ] 2 we denote by α ( i ) its i -th index and by x α the pair ( x α (1) , x α (2) ) . For variables x 1 , . . . , x n , we then write χ α ( x ) for the monomial ∏ i ∈ α x i . Given a predicate P : {-1 , +1 } 2 →{ 0 , 1 } , an instance I of the CSP(P) problem over variables x 1 , . . . , x n is a multi-set of triplets ( w,c, α ) representing constraints of the form P ( c 1 x α (1) , c 2 x α (2) ) = 1 where α ∈ [ n ] 2 is the scope, c ∈ {± 1 } 2 is the negation pattern and w ≥ 0 is the weight of the constraint. For brevity we often write P ( c ◦ x α ) in place of P ( c 1 x α (1) , c 2 x α (2) ) . We let W = ∑ ( w,c,α ) ∈I w. We can represent the predicate P as the multilinear polynomial of degree 2 in indeterminates x α (1) , x α (2) ,

where ˆ p ( α ′ ) is the coefficient in P of the monomial χ α ′ ( x ) . Notice that this formulation does not rule out predicates with the same multi-index but different negation patterns or multi-indices in which an index appears multiple times. Given a predicate P , an instance I of CSP(P) with m constraints and x ∈ {± 1 } n , we define

For instance, MAXCUT on a graph G = ([ n ] , E ) can be captured in this framework where P ( x, y ) = (1 -xy ) / 2 , each edge ( i, j ) ∈ [ n ] 2 yields a triplet ( w,c, α ) where w = 1 , c = (1 , 1) and α = ( i, j ) .

In the noisy prediction model for CSPs, for an instance I of CSP(P), we assume there is some fixed assigment x ∗ with value val I ( x ∗ ) = opt I . The algorithm has access to a prediction vector

Y ∈ {± 1 } n such that predictions y i 's are 2 -wise independently correct with probability 1+ ε 2 for unknown bias ε . We let Z = Y 2 ε . With a slight abuse of notation we also write P ( c ◦ Z α ) even though Z is a rescaled boolean vector.

For a literal i ∈ [ n ] and an instance I of CSP(P) we let S i := { ( w,c, α ) ∈ I | α (1) = i } . As in Section 3.1.1, we can define ∆ -wide literals and instances. For an instance I , we partition the constraints in S i into two sets: the ∆ -prefix for i comprises the ∆ heaviest constraints in S i (breaking ties arbitrarily), while the remaining constraints make up the ∆ -suffix for i , which we denote by ˜ S i . We fix a parameter η ∈ (0 , 1 / 2 ) . We let W i = ∑ ( w,c,α ) ∈S i w i .

Definition 5.1 ( ∆ -Narrow/Wide) . A literal i is ∆ -wide if the total weight of its in its ∆ -prefix is at most ηW i , and so the weight of edges in its ∆ -suffix is at least (1 -η ) W i . Otherwise, the literal i is ∆ -narrow . An instance I of CSP(P) is ∆ -wide if ∑ i ∈ [ n ] ∆ -wide W i ≥ (1 -η ) W .

We are now ready to state the main theorem of the section.

Theorem 5.2. Let P : {± 1 } 2 → { 0 , 1 } be a predicate. Let ε ′ ∈ (0 , 1) , η ∈ (0 , 1 / 2) and ∆ ≥ O (1 / ( ε ′ · ε ) 2 ) . There exists a polynomial-time randomized algorithm that, given a ∆ -wide I in CSP(P) and noisy predictions with bias ε , returns a vector ˆ x ∈ {± 1 } n satisfying

with probability at least 0 . 98 .

The proof of Theorem 5.2 follows closely that of Theorem 3.4, and is deferred to Appendix B.2.

6 Closing Remarks

Our work suggests many directions for future research. One immediate question is to quantitatively improve the exponent of ε for noisy predictions, and the constants. Here are some broader questions:

We assume that our noisy predictions are correct with probability equal to 1 / 2 + ε ; we can easily extend to the case where each node has a prediction that is correct with some probability 1 / 2 + ε i , and each ε i ∈ Θ( ε ) . Can we extend to the case when we are only guaranteed ε i ≥ ε for every i ?

For which other problems can we improve the performance of the state-of-the-art algorithms using noisy predictions? As we showed, the ideas used for the ∆ -wide case extend to more general maximization problems on 2 -CSPs with 'high degree', but can we extend the results for the 'low-degree' case where each variable does not have a high-enough degree to infer a clear signal? Can we extend these to minimization versions of 2 -CSPs?

What general lower bounds can we give for our prediction models? We feel that α GW + O ( ε ) is a natural barrier. One 'evidence' we have is the following integrality gap for the SDP used in the partial information model; starting from a gap instance and an SDP solution exhibiting opt ≤ α GW · sdp for the standard SDP (without incorporating revealed information), given labels for an εn vertices, our new SDP simply fixes the positions of the corresponding εn vectors, but doing that from the given SDP solution decreases the SDP value by at most O ( ε ) in expectation, which still yields opt ≤ ( α GW + O ( ε ))sdp . (Note that you can replace the SDP gap with any hypothetical gap instance for stronger relaxations.)

Given that the partial predictions model is easier than the noisy predictions model and our entire algorithm for the partial model is based on this SDP, this might be considered as a convincing lower bound, but it would be nicer to have more general lower bounds against all polynomial-time algorithms.

Our models only assume pairwise independence between vertices: can we extend our results to other ways of modeling correlations between the predictions? In addition to stochastic predictions, can we incorporate geometric predictions (e.g., in random graph models where the probability of edges depend on the proximity of the nodes)?

Acknowledgments

We thank Ola Svensson for enjoyable discussions. AG was supported in part by NSF awards CCF-2006953 and CCF-2224718, and by Google, Inc. EL was supported in part by NSF award CCF-2236669 and by Google, Inc. DP was supported in part by NSF awards CCF-1955703 and CCF-2329230, and by Google, Inc.

References

Online index for algorithms with predictions, 2023. https://algorithms-with-predictions. github.io/ .

Nir Ailon, Anup Bhattacharya, Ragesh Jaiswal, and Amit Kumar. Approximate clustering with same-cluster queries. In Anna R. Karlin, editor, 9th Innovations in Theoretical Computer Science Conference, ITCS 2018, January 11-14, 2018, Cambridge, MA, USA , volume 94 of LIPIcs , pages 40:1-40:21. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2018.

Matteo Almanza, Flavio Chierichetti, Silvio Lattanzi, Alessandro Panconesi, and Giuseppe Re. Online facility location with multiple advice. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 4661-4673, 2021.

Keerti Anand, Rong Ge, and Debmalya Panigrahi. Customizing ML predictions for online algorithms. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , volume 119 of Proceedings of Machine Learning Research , pages 303-313. PMLR, 2020.

Keerti Anand, Rong Ge, Amit Kumar, and Debmalya Panigrahi. Online algorithms with multiple predictions. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , volume 162 of Proceedings of Machine Learning Research , pages 582-598. PMLR, 2022.

Antonios Antoniadis, Christian Coester, Marek Eliás, Adam Polak, and Bertrand Simon. Online metric algorithms with untrusted predictions. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , volume 119 of Proceedings of Machine Learning Research , pages 345-355. PMLR, 2020a.

Antonios Antoniadis, Themis Gouleakis, Pieter Kleer, and Pavel Kolev. Secretary and online matching problems with machine learned advice. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , 2020b.

Antonios Antoniadis, Christian Coester, Marek Eliás, Adam Polak, and Bertrand Simon. Mixing predictions for online metric algorithms. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , volume 202 of Proceedings of Machine Learning Research , pages 969-983. PMLR, 2023.

Sanjeev Arora, David R. Karger, and Marek Karpinski. Polynomial time approximation schemes for dense instances of np-hard problems. J. Comput. Syst. Sci. , 58(1):193-210, 1999.

Hassan Ashtiani, Shrinu Kushagra, and Shai Ben-David. Clustering with same-cluster queries. Advances in neural information processing systems , 29, 2016.

Yossi Azar, Stefano Leonardi, and Noam Touitou. Flow time scheduling with uncertain processing time. In Samir Khuller and Virginia Vassilevska Williams, editors, STOC '21: 53rd Annual ACM SIGACT Symposium on Theory of Computing, Virtual Event, Italy, June 21-25, 2021 , pages 1070-1080. ACM, 2021.

Yossi Azar, Debmalya Panigrahi, and Noam Touitou. Online graph algorithms with predictions. In Joseph (Seffi) Naor and Niv Buchbinder, editors, Proceedings of the 2022 ACM-SIAM Symposium on Discrete Algorithms, SODA 2022, Virtual Conference / Alexandria, VA, USA, January 9 - 12, 2022 , pages 35-66. SIAM, 2022.

Étienne Bamas, Andreas Maggiori, Lars Rohwedder, and Ola Svensson. Learning augmented energy minimization via speed scaling. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, MariaFlorina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , 2020a.

Étienne Bamas, Andreas Maggiori, and Ola Svensson. The primal-dual method for learning augmented algorithms. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , 2020b.

Evripidis Bampis, Bruno Escoffier, and Michalis Xefteris. Parsimonious learning-augmented approximations for dense instances of NP -hard problems, 2024.

Nikhil Bansal, Christian Coester, Ravi Kumar, Manish Purohit, and Erik Vee. Learning-augmented weighted paging. In Joseph (Seffi) Naor and Niv Buchbinder, editors, Proceedings of the 2022 ACM-SIAM Symposium on Discrete Algorithms, SODA 2022, Virtual Conference / Alexandria, VA, USA, January 9 - 12, 2022 , pages 67-89. SIAM, 2022.

Giulia Bernardini, Alexander Lindermayr, Alberto Marchetti-Spaccamela, Nicole Megow, Leen Stougie, and Michelle Sweering. A universal error measure for input predictions applied to online graph problems. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022 , 2022.

Ilan Reuven Cohen and Debmalya Panigrahi. A general framework for learning-augmented online allocation. In Kousha Etessami, Uriel Feige, and Gabriele Puppis, editors, 50th International Colloquium on Automata, Languages, and Programming, ICALP 2023, July 10-14, 2023, Paderborn, Germany , volume 261 of LIPIcs , pages 43:1-43:21. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2023.

Alberto Del Pia, Mingchen Ma, and Christos Tzamos. Clustering with queries under semi-random noise. In Conference on Learning Theory , pages 5278-5313. PMLR, 2022.

Michael Dinitz, Sungjin Im, Thomas Lavastida, Benjamin Moseley, and Sergei Vassilvitskii. Faster matchings via learned duals. In Marc'Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual , pages 10393-10406, 2021.

Paul Dütting, Silvio Lattanzi, Renato Paes Leme, and Sergei Vassilvitskii. Secretaries with advice. In Péter Biró, Shuchi Chawla, and Federico Echenique, editors, EC '21: The 22nd ACM Conference on Economics and Computation, Budapest, Hungary, July 18-23, 2021 , pages 409-429. ACM, 2021.

Jon C. Ergun, Zhili Feng, Sandeep Silwal, David P. Woodruff, and Samson Zhou. Learning-augmented $k$-means clustering. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022.

Uriel Feige, Marek Karpinski, and Michael Langberg. Improved approximation of max-cut on graphs of bounded degree. J. Algorithms , 43(2):201-219, 2002.

Dimitris Fotakis, Evangelia Gergatsouli, Themis Gouleakis, and Nikolas Patris. Learning augmented online facility location. CoRR , abs/2107.08277, 2021. URL https://arxiv.org/abs/2107. 08277 .

Buddhima Gamlath, Silvio Lattanzi, Ashkan Norouzi-Fard, and Ola Svensson. Approximate cluster recovery from noisy labels. In Po-Ling Loh and Maxim Raginsky, editors, Conference on Learning Theory, 2-5 July 2022, London, UK , volume 178 of Proceedings of Machine Learning Research , pages 1463-1509. PMLR, 2022.

Suprovat Ghoshal, Konstantin Makarychev, and Yury Makarychev. Constraint satisfaction problems with advice. CoRR , abs/2403.02212, 2024. doi: 10.48550/ARXIV.2403.02212. URL https: //doi.org/10.48550/arXiv.2403.02212 .

Michel X. Goemans and David P. Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. ACM , 42(6):1115-1145, 1995.

Sreenivas Gollapudi and Debmalya Panigrahi. Online algorithms for rent-or-buy with expert advice. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA , volume 97 of Proceedings of Machine Learning Research , pages 2319-2327. PMLR, 2019.

Anupam Gupta, Debmalya Panigrahi, Bernardo Subercaseaux, and Kevin Sun. Augmenting online algorithms with ε -accurate predictions. In NeurIPS , Dec 2022.

Jun-Ting Hsieh and Pravesh K. Kothari. Approximating max-cut on bounded degree graphs: Tighter analysis of the FKL algorithm. CoRR , abs/2206.09204, 2022.

Shaofeng H.-C. Jiang, Erzhi Liu, You Lyu, Zhihao Gavin Tang, and Yubo Zhang. Online facility location with predictions. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022a.

Zhihao Jiang, Debmalya Panigrahi, and Kevin Sun. Online algorithms for weighted paging with predictions. ACM Trans. Algorithms , 18(4):39:1-39:27, 2022b.

Kasper Green Larsen, Michael Mitzenmacher, and Charalampos E. Tsourakakis. Clustering with a faulty oracle. In Yennun Huang, Irwin King, Tie-Yan Liu, and Maarten van Steen, editors, WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020 , pages 2831-2834. ACM / IW3C2, 2020.

Alexandra Anna Lassota, Alexander Lindermayr, Nicole Megow, and Jens Schlöter. Minimalistic predictions to schedule jobs with online precedence constraints. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , volume 202 of Proceedings of Machine Learning Research , pages 18563-18583. PMLR, 2023.

Silvio Lattanzi, Thomas Lavastida, Benjamin Moseley, and Sergei Vassilvitskii. Online scheduling via learned weights. In Shuchi Chawla, editor, Proceedings of the 2020 ACM-SIAM Symposium on Discrete Algorithms, SODA 2020, Salt Lake City, UT, USA, January 5-8, 2020 , pages 1859-1877. SIAM, 2020.

Thomas Lavastida, Benjamin Moseley, R. Ravi, and Chenyang Xu. Learnable and instance-robust predictions for online matching, flows and load balancing. In Petra Mutzel, Rasmus Pagh, and Grzegorz Herman, editors, 29th Annual European Symposium on Algorithms, ESA 2021, September 6-8, 2021, Lisbon, Portugal (Virtual Conference) , volume 204 of LIPIcs , pages 59:1-59:17. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021.

Alexander Lindermayr, Nicole Megow, and Martin Rapp. Speed-oblivious online scheduling: Knowing (precise) speeds is not necessary. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , volume 202 of Proceedings of Machine Learning Research , pages 21312-21334. PMLR, 2023.

Thodoris Lykouris and Sergei Vassilvitskii. Competitive caching with machine learned advice. J. ACM , 68(4):24:1-24:25, 2021.

Arya Mazumdar and Barna Saha. Clustering with noisy queries. Advances in Neural Information Processing Systems , 30, 2017.

Michael Mitzenmacher. Scheduling with predictions and the price of misprediction. In Thomas Vidick, editor, 11th Innovations in Theoretical Computer Science Conference, ITCS 2020, January 12-14, 2020, Seattle, Washington, USA , volume 151 of LIPIcs , pages 14:1-14:18. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020.

Michael Mitzenmacher and Sergei Vassilvitskii. Algorithms with predictions. In Tim Roughgarden, editor, Beyond the Worst-Case Analysis of Algorithms , pages 646-662. Cambridge University Press, 2020.

Michael Mitzenmacher and Sergei Vassilvitskii. Algorithms with predictions. Commun. ACM , 65(7): 33-35, 2022.

Ryan O'Donnell. Analysis of boolean functions . Cambridge University Press, 2014.

Manish Purohit, Zoya Svitkina, and Ravi Kumar. Improving online algorithms via ML predictions. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , pages 9684-9693, 2018.

Prasad Raghavendra and Ning Tan. Approximating CSPs with global cardinality constraints using sdp hierarchies. In Proceedings of the twenty-third annual ACM-SIAM symposium on Discrete Algorithms , pages 373-387. SIAM, 2012.

Tim Roughgarden, editor. Beyond the Worst-Case Analysis of Algorithms . Cambridge University Press, 2020.

Alexander Wei. Better and simpler learning-augmented online caching. In Jaroslaw Byrka and Raghu Meka, editors, Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques, APPROX/RANDOM 2020, August 17-19, 2020, Virtual Conference , volume 176 of LIPIcs , pages 60:1-60:17. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2020.

Alexander Wei and Fred Zhang. Optimal robustness-consistency trade-offs for learning-augmented online algorithms. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual , 2020.

Appendix

A Missing proofs for MAXCUT in the partial prediction model

Proof of Theorem 4.1. Given a graph G = ( V, E ) with the optimal cut ( A ∗ , B ∗ ) that cuts E ∗ = E ∩ E ( A ∗ , B ∗ ) , let S be the set of vertices whose label is given, and let A = A ∗ ∪ S , B = B ∗ ∪ S . Consider the following MAXCUT SDP that fixes the vertices with the revealed labels.

Note that this is still a valid relaxation so the optimal SDP value sdp is at least opt . For each edge e ∈ E ∗ , e ∈ E ( A,B ) with probability ε 2 ; in other words, both of its endpoints will reveal their labels. Let τ denotes the total weight of such edges, so that E [ τ ] = ε 2 opt . Note that sdp ≥ opt for every partial prediction.

Perform the standard hyperplane rounding. For each e ∈ E ∗ ∩ E ( A,B ) , the rounding will always cut e . For all other edges, we have an approximation ratio of α GW. Therefore, the expected weight of the cut edges is at least

Proof of Theorem 4.2. Given a graph G = ( V, E ) with the optimal cut ( A ∗ , B ∗ ) that cuts E ∗ = E ∩ E ( A ∗ , B ∗ ) , let S be the set of vertices whose label is given, and let A = A ∗ ∪ S , B = B ∗ ∪ S . Let E ′ be the set of the edges that are incident on A or B . Each edge cut in the optimal solution will be in E ′ with probability 2 ε -ε 2 . Let τ be the total weight of the edges in E ∗ ∩ E ′ so that E [ τ ] = (2 ε -ε 2 )opt . Guess the value of τ (up to a o (1) multiplicative error that we will ignore in the proof), and consider the following MAXCUT SDP that fixes the vertices with the revealed labels and requires a large SDP contribution from E ′ .

Given the correctly guessed value of τ , the optimal solution is still feasible for the above SDP, so sdp ≥ opt . We use Raghavendra and Tan [2012]'s rounding algorithm, which is briefly recalled below.

For each i ∈ [ n ] , define µ i ∈ [ -1 , +1] and w i ∈ R n such that v i = µ i v 0 + w i and w i ⊥ v 0 . Let w i = w i / ∥ w i ∥ . ( w i = 0 if and only if v i = ± v 0 . Then define w i = 0 .)

Pick a random Gaussian vector g orthogonal to v 0 . Let ξ i := ⟨ g, w i ⟩ . Note that each ξ i is a standard Gaussian.

Let the threshold t i := Φ -1 ( µ i / 2 + 1 / 2) where Φ is the CDF of a standard Gaussian.

If ξ i ≤ t i , set x i = 1 and otherwise set x i = -1 .

Raghavendra and Tan showed that this rounding achieves an ( α RT ≈ 0 . 858) -approximation for every edge.

Consider an edge ( i, j ) ∈ E ′ and without loss of generality, assume i ∈ B , which implies that v i = -v 0 . The contribution of this edge to the SDP objective is µ j / 2 + 1 / 2 . Note that Pr[ x j = 1] is exactly µ j / 2 + 1 / 2 and E [ x j ] = ( µ j / 2 + 1 / 2) -(1 / 2 -µ j / 2) = µ j . So, we get a 1 -approximation from this edge. Since other edges still have an α RT-approximation, the total expected weight of the edges cut is at least

Hence the proof of Theorem 4.2.

B Missing details for 2-CSPs

The goal of this section is to establish Theorem 5.2 for dense instances of 2 -CSPs.

B.1 The 2 -CSP Algorithm for Dense Instances

First observe that we may assume without loss of generality that each ( w,c, α ) appears exactly twice in I . This is convenient so that for all x ∈ {± 1 } n , val I ( x ) = ∑ i ∈ [ n ] ∑ ( w,c,α ) ∈ S i w · P ( c ◦ x α ) . With a slight abuse of notation, for all ( w,c, α ) ∈ S i , we let

̸

and

̸

We further define ˜ S i ⊆ S i to be subset of constraints in S i that are not part of the ∆ -prefix of i . We can now state the algorithm behind Theorem 5.2, which amounts to the following two steps.

Solve the linear program

subject to

for some large enough absolute constant C > 0 . Let ˆ x ∈ [ -1 , +1] n be the found optimal solution.

Repeat O (1 /η ) times independently and output the best assignment X ∗ : independently for each i ∈ [ n ] set X i = 1 with probability (1 + ˆ x i ) / 2 and X i = -1 otherwise.

The LP above generalize the one in eq. (1), which comes as a special case where ˆ p α ′ = 0 for all α ′ ⊂ α ∈ [ n ] 2 . Indeed, since predicates contain only two literals, the program is linear.

B.2 The Analysis of the 2 -CSP Algorithm

We obtain here the proof of Theorem 5.2.

Feasibility of the best assignment As in Lemma 3.5, we first prove that, in expectation over the prediction Y , x ∗ is a feasible solution to the program.

Lemma B.1. Consider the settings of Theorem 5.2. Then

Proof. First, by definition of ∆ -wide instance,

Second, by definition for any ∆ -wide vertex i ,

Hence it remains to show

Now, recall that E [ Y i ] = 2 εx ∗ i and thus E [ Z ] = x ∗ . So for any ( c, α ) ∈ I , E [ P ( c ◦ Z α )] = E [ P ( c ◦ x ∗ α )] by pair-wise independence of the predictions. Thus it suffices to study, for each ∆ -wide i , var ( ∑ ( w,c,α ) ∈ S i wP ( c ◦ Z α \ i ) ) . To this end, notice that for any α, α ′ ∈ S i with α ∩ α ′ = { i } it holds

Moreover, since | α | = 2 , there are at most 4 distinct negation patterns. Therefore, by the AM-GM inequality

where we used the fact that entries of Z are bounded by 1 /ε and the coefficients of a boolean predicate are bounded by 1 (by Parseval's Theorem, see O'Donnell [2014]). By construction of ˜ S i , each ( w,c, α ) ∈ ˜ S i must satisfy w ≤ W i / ∆ . Using Holder's inequality

We can use this bound on the variance in combination with Chebishev's inequality to obtain, for λ > 0 ,

Let λ := O ( W i / ( ε √ ∆)) . A peeling argument now completes the proof:

Analysis of the algorithm We can use Lemma B.1 to obtain our main theorem for CSPs.

Proof of Theorem 5.2. We follow closely the proof of Lemma 3.6. Consider one of the assignments X ∈ {± 1 } n found in the second step of the algorithm. Recall ˆ x ∈ [ -1 , +1] n denotes the optimal fractional solution found by the algorithm. We may rewrite for each ∆ -wide i and ( w,c, α ) ∈ ˜ S i

We bound each term in Equation (11) separately. First, notice that by Markov's inequality and Lemma B.1, with probability 0 . 99 , x ∗ is a feasible solution to the LP. Conditioning on this event E

By Holder's inequality and the fact that x ∗ is feasible, for ∆ -wide i ,

Since by construction ˆ x is feasible, another application of Holder's inequality also yields the following bound on the third term,

For the second term in Equation (11), by construction of X we have E [ P ( c ◦ X α ) | E ] = P ( c ◦ ˆ x α ) . Combining the three bounds, we get that

Applying Markov's inequality on the random variable opt I -1 W ∑ i ∈ [ n ] ∆ -wide ∑ ( w,c,α ) ∈ ˜ S i wP ( c ◦ X α ) , we get

The theorem follows since we sample O (1 /η ) independent assignments X and pick the best.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Theorems 3.1, 4.1 and 5.2, which are proved in the paper, formalize the claims in the abstract.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The models' assumptions are accurately described (see Section 2).

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All assumptions are clearly stated. All theorems are formally proved in the main body or in the appendix.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: The paper does not include experiments.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: The paper does not include experiments requiring code.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: The paper does not include experiments.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: The paper does not include experiments.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: The paper does not include experiments.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This is a theoretical work; we do not foresee any societal impact of this work.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper does not use existing assets.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFor the Max Cut Problem, the authors first consider predictions that are independently correct with probability 1/2 + $\\epsilon$ (noisy prediction model). They obtain an approximation guarantee of $0.878 + \\Omega(\\epsilon^4)$, improving upon best-known approximation guarantees. The idea of the algorithm is to divide instances by wide and narrow graphs, a notion related to high-degree and low-degree graphs. By obtaining approximation guarantees for both cases and optimizing over some parameters, they obtain their first main result.\nIn the second setting, they consider a partial prediction model, in which one gets a correct prediction, but only for an $\\epsilon$-fraction of randomly chosen vertices. In this case they obtain a $0.858 + \\Omega(\\epsilon)$ approximation algorithm, which is slightly worse than the $0.878$-approximation for small $\\epsilon$, but in general the dependency on $\\epsilon$ is more favorable. The slightly worse constant is because one needs to employ a rounding scheme from Raghavendra and Tan, which has this approximation guarantee. However, by doing this, one obtains a better dependency on $\\epsilon$.\nFinally, the authors also consider 2-CSPs in the noisy prediction model. The ideas are similar to the case for the Max Cut Problem.\n\nSTRENGTHS:\nThe paper is well-written and the proofs seem sound. The results are nice and improve upon state-of-the-art approximation algorithms given good predictions on the solution. To the best of my knowledge, this paper is the first to substantially contribute towards learning-augmented algorithms in the area of approximation algorithms. Hence, the paper is innovative and might lead to new results in this interesting area.\n\nWEAKNESSES:\nThe results are specifically tailored for the problems considered in this paper. However, this is no surprise as this paper is the first to study learning-augmented algorithms in this area. Therefore, I do not see this as a strong weakness.\n\nQUESTIONS:\n-\n\nLIMITATIONS:\nIn my opinion, all limitations have been properly addressed.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors study algorithms constraint satisfaction problems, in particular, MAX-CUT\nwhich are provided with predictions mildly correlated with the optimal solution.\nIn the case of MAX-CUT, we have a prediction +1 or -1 for each vertex of the graph\nwhich suggests which side of the maximum cut should it belong to.\nFor each vertex, this prediction is correct with probability 0.5+eps,\nassuming a pairwise independence between the predictions.\nThey call this eps-correlated predictions.\nThey also consider a similar model where all predictions are guaranteed to be correct\nbut they are available only for eps-fraction of the vertices chosen randomly.\n\nTheir main result is a poly-time algorithm for MAX-CUT which, given\neps-correlated predictions, achieves an improvement of order Omega(eps^4)\nover the best possible approximation ratio achievable by a poly-time algorithm\nwithout any predictions.\nTheir algorithm is based on combination of linear programming and semidefinite\nprogramming techniques.\n\nSTRENGTHS:\n* I believe that the problem of solving MAX-CUT\nstarting with a solution which is mildly correlated with the optimum is a basic\nquestion about SDP techniques for MAX-CUT. Since MAX-CUT is one of the central\nproblems in theory of algorithms, I consider the progress on this question to\nbe the main strength of this paper.\n\n* the main clear model of MAX-CUT with eps-correlated predictions is also extended\nto CSPs and also to the model with partial predictions.\n\nWEAKNESSES:\nNothing particular.\n\nQUESTIONS:\n* Is the precision parameter of the predictions epsilon known to the algorithm?\n\n* Many works, especially for online problems, consider predictions whose error\nis distributed adversarially. Do you think such predictions are also applicable\nto MAX-CUT and CSPs?\n\nLIMITATIONS:\nproperly stated in the theorem statements.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n**Problem Studied**\n\nThis paper studies the Max Cut and 2-CSP problems in a setting where there is some noisy prediction of the optimal solution. In particular, the paper considers the following three settings:\n- Max Cut in the \"noisy prediction model\": Here, each vertex gives its true label with probability $\\frac{1}{2} + \\epsilon$ (and its opposite label otherwise). \n- Max Cut in the \"partial prediction model\": Here, an $\\epsilon$-fraction of the vertices reveal their true label at random.\n- 2-CSPs in the noisy prediction model.\n\n**Main Results / Contributions**\n\nFor Max Cut, the main result is that it is possible to beat the Goemans-Williamson approximation factor when $\\epsilon > 0$. For the noisy prediction model, the authors obtain an approximation ratio that is roughly $\\alpha_{GW} +  \\Omega(\\epsilon^4)$. For the partial prediction model, it is possible to get $\\alpha_{GW} + \\Omega(\\epsilon^2)$, or $\\alpha_{GT} + \\Omega(\\epsilon)$, where $\\alpha_{GT} \\approx 0.858$ is the approximation ratio of the algorithm by Raghavendra and Tan.\n\nThe authors extend one result to 2-CSPs in the noisy prediction model that are \"wide\".\n\nSTRENGTHS:\nThe paper is very well written. It is nice to read and the contributions of the paper are clear. It studies a natural variant (prediction-augmented) of a well-known problem (max cut). It also answers an open question posed by Ola Svensson, which is a good sign.\n\nWEAKNESSES:\nThe title is perhaps promising more than the paper delivers -- the paper only studies Max Cut and a subclass of 2-CSP instances, so it is a bit misleading to use \"maximization problems\" in the title.\n\nQUESTIONS:\n1. For the noisy prediction model of Max Cut, a natural algorithm that comes to mind is to return the better of the Goemans-Williamson cut and the predicted cut. Is there an example where this algorithm does not do better than $\\alpha_{GW}$?\n\n2. In the literature on algorithms with predictions, one common way people have modeled the \"prediction\" is as a single predicted solution, instead of assuming it is generated from a distribution. They then usually parameterize the performance of the algorithm (e.g. approximation ratio), as a function of the \"error\" in the (single) predicted solution, where \"error\" is something that has to be defined depending on the problem. This has the advantage of not needing to define the form of the distribution that the prediction comes from (like the noisy prediction model or the partial prediction model), and gives a somewhat more \"instance-dependent\" guarantee. What are your thoughts on this way of viewing the prediction in the context of your problem? Do you think it is a better or worse model, and why?\n\n3. As $\\epsilon \\to \\frac{1}{2}$ in the noisy predictions model, the prediction approaches the optimal solution, and so the approximation ratio of the algorithm should intuitively go to 1. Is this the case for the algorithm considered in this paper? This is not clear from the theorem statement because it has a big-Omega in the bound. Is there is a way to write the approximation ratio exactly in terms of $\\epsilon$ to show that it approaches 1 as $\\epsilon \\to \\frac{1}{2}$? If so, I think that would be instructive. Similarly for the partial predictions model as $\\epsilon \\to 1$. \n\n4. What is the intuition for why predictions are not needed for narrow instances?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors discuss about a setup if the approximation ratio of the known approximation algorithms for offline NP-hard problems can be improved in the cases where we have access to noisy or partial predictions. They answer this investigation positively for MaxCut and Constraint Satisfaction (CSP) problems. Their theoretical investigation proves that the worst-case performance for the MaxCut and 2-CSP problems can be improved with assuming access to pairwise independent predictions that are correct with the probability $1/2 + \\epsilon$. This corresponds to the noisy predictions setting. In the partial prediction settings, they assume that each vertex reveals their correct label with probability $\\epsilon$ pairwise independently. Similarly, this theoretically improves the worst-case performance for the MaxCut problem.\n\nSTRENGTHS:\nThe central motivation of the paper is quite interesting. With the abundance of the prediction ML models, utilizing their outputs in improving the approximation guarantees of famous combinatorial optimization problems makes it a challenging discussion. The problem setups are explained clearly. Considering much of the related work focuses on the online setting, making contributions for the offline setting seems significant.\n\nWEAKNESSES:\n- The introduction could motivate the readers more about the significance of the offline setting. \n- On the first glance, it is not very clear how this papers differs from the concurrent work mentioned on the last paragraph of the Related Work (besides one of them making the full independence assumption between the predictions).\n- The existence of predictions is treated as a given but often times acquiring those predictions comes with a significant cost.\n\nQUESTIONS:\nMy questions are in parallel with the weaknesses I listed above:\n- Could you please motivate more about the advantages of utilizing predictions in the offline setting with practical applications?\n- How does your setup differ from the works of [Bampis et al., 2024] and [Ghoshal et al., 2024]?\n- Do you think it is possible to define a computational upper bound on the cost of getting the predictions so that an approximation algorithm can still be considered polynomial? In other words, can there be a trade-off between the cost of predictions and the improvement on the worst-case bound?\n\nLIMITATIONS:\nLimitations of the paper are not explicitly discussed beyond the assumptions made in Section 2.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}