Title: Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Method
Venue: NeurIPS 2023
Accepted: yes
Keywords: Policy Gradient

Combinatorial Optimization

Gradient Descent
Abstract: Deep Neural Networks and Reinforcement Learning methods have empirically shown great promise in tackling challenging combinatorial problems. In those methods a deep neural network is used as a solution generator which is then trained by gradient-based methods (e.g., policy gradient) to successively obtain better solution distributions.
In this work we introduce a novel theoretical framework for analyzing the effectiveness of such methods. We ask whether there exist generative models that (i) are expressive enough to generate approximately optimal solutions; (ii) have a tractable, i.e, polynomial in the size of the input, number of parameters; (iii) their optimization landscape is benign in the sense that it does not contain sub-optimal stationary points. Our main contribution is a positive answer to this question. Our result holds for a broad class of combinatorial problems including Max- and Min-Cut, Max-$k$-CSP, Maximum-Weight-Bipartite-Matching, and the Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla gradient descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.

Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods

Abstract

Deep Neural Networks and Reinforcement Learning methods have empirically shown great promise in tackling challenging combinatorial problems. In those methods a deep neural network is used as a solution generator which is then trained by gradient-based methods (e.g., policy gradient) to successively obtain better solution distributions. In this work we introduce a novel theoretical framework for analyzing the effectiveness of such methods. We ask whether there exist generative models that (i) are expressive enough to generate approximately optimal solutions; (ii) have a tractable, i.e, polynomial in the size of the input, number of parameters; (iii) their optimization landscape is benign in the sense that it does not contain sub-optimal stationary points. Our main contribution is a positive answer to this question. Our result holds for a broad class of combinatorial problems including Max- and Min-Cut, Max𝑘 -CSP, Maximum-Weight-Bipartite-Matching, and the Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla gradient descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.

1 Introduction

Gradient descent has proven remarkably effective for diverse optimization problems in neural networks. From the early days of neural networks, this has motivated their use for combinatorial optimization [HT85, Smi99, VFJ15, BPL + 16]. More recently, an approach by [BPL + 16], where a neural network is used to generate (sample) solutions for the combinatorial problem. The parameters of the neural network thus parameterize the space of distributions. This allows one to perform gradient steps in this distribution space. In several interesting settings, including the Traveling Salesman Problem, they have shown that this approach works remarkably well. Given the widespread application but also the notorious difficulty of combinatorial optimization [GLS12, PS98, S + 03, Sch05, CLS + 95], approaches that provide a more general solution framework are appealing.

This is the point of departure of this paper. We investigate whether gradient descent can succeed in a general setting that encompasses the problems studied in [BPL + 16]. This requires a parameterization

Dimitris Fotakis

Vasilis Kontonis

of distributions over solutions with a 'nice' optimization landscape (intuitively, that gradient descent does not get stuck in local minima or points of vanishing gradient) and that has a polynomial number of parameters. Satisfying both requirements simultaneously is non-trivial. As we show precisely below, a simple lifting to the exponential-size probability simplex on all solutions guarantees convexity; and, on the other hand, compressed parameterizations with 'bad' optimization landscapes are also easy to come by (we give a natural example for Max-Cut in Remark 1). Hence, we seek to understand the parametric complexity of gradient-based methods, i.e., how many parameters suffice for a benign optimization landscape in the sense that it does not contain 'bad' stationary points.

We thus theoretically investigate whether there exist solution generators with a tractable number of parameters that are also efficiently optimizable, i.e., gradient descent requires a small number of steps to reach a near-optimal solution. We provide a positive answer under general assumptions and specialize our results for several classes of hard and easy combinatorial optimization problems, including Max-Cut and Min-Cut, Max𝑘 -CSP, Maximum-Weighted-Bipartite-Matching and Traveling Salesman. We remark that a key difference between (computationally) easy and hard problems is not the ability to find a compressed and efficiently optimizable generative model but rather the ability to efficiently draw samples from the parameterized distributions.

1.1 Our Framework

We introduce a theoretical framework for analyzing the effectiveness of gradient-based methods on the optimization of solution generators in combinatorial optimization, inspired by [BPL + 16].

Let ℐ be a collection of instances of a combinatorial problem with common solution space 𝑆 and 𝐿 ( · ; 𝐼 ) : 𝑆 → R be the cost function associated with an instance 𝐼 ∈ ℐ , i.e., 𝐿 ( 𝑠 ; 𝐼 ) is the cost of solution 𝑠 given the instance 𝐼 . For example, for the Max-Cut problem the collection of instances ℐ corresponds to all graphs with 𝑛 nodes, the solution space 𝑆 consists of all subsets of nodes, and the loss 𝐿 ( 𝑠 ; 𝐼 ) is equal to (minus) the weight of the cut ( 𝑠, [ 𝑛 ] ∖ 𝑠 ) corresponding to the subset of nodes 𝑠 ∈ 𝑆 (our goal is to minimize 𝐿 ).

Definition 1 (Solution Cost Oracle) . For a given instance 𝐼 we assume that we have access to an oracle 𝒪 ( · ; 𝐼 ) to the cost of any given solution 𝑠 ∈ 𝑆 , i.e., 𝒪 ( 𝑠 ; 𝐼 ) = 𝐿 ( 𝑠 ; 𝐼 ) .

The above oracle is standard in combinatorial optimization and query-efficient algorithms are provided for various problems [RSW17, GPRW19, LSZ21, AEG + 22, PRW22]. We remark that the goal of this work is not to design algorithms that solve combinatorial problems using access to the solution cost oracle (as the aforementioned works do). This paper focuses on landscape design: the algorithm is fixed , namely (stochastic) gradient descent; the question is how to design a generative model that has a small number of parameters and the induced optimization landscape allows gradient-based methods to converge to the optimal solution without getting trapped at local minima or vanishing gradient points.

Let ℛ be some prior distribution over the instance space ℐ and 𝒲 be the space of parameters of the model. We now define the class of solution generators. The solution generator 𝑝 ( 𝑤 ) with parameter 𝑤 ∈ 𝒲 takes as input an instance 𝐼 and generates a random solution 𝑠 in 𝑆 . To distinguish between the output, the input, and the parameter of the solution generator, we use the notation 𝑝 ( · ; 𝐼 ; 𝑤 ) to denote the distribution over solutions and 𝑝 ( 𝑠 ; 𝐼 ; 𝑤 ) to denote the probability of an individual solution 𝑠 ∈ 𝑆 . We denote by 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} the above parametric class of solution generators. For some parameter 𝑤 , the loss corresponding to the solutions sampled by 𝑝 ( · ; 𝐼 ; 𝑤 ) is equal to

Our goal is to optimize the parameter 𝑤 ∈ 𝒲 in order to find a sampler 𝑝 ( · ; 𝐼 ; 𝑤 ) whose loss ℒ ( 𝑤 ) is close to the expected optimal value opt :

The policy gradient method [Kak01] expresses the gradient of ℒ as follows

and updates the parameter 𝑤 using the gradient descent update. Observe that a (stochastic) policy gradient update can be implemented using only access to a solution cost oracle of Definition 1.

Solution Generators. In [BPL + 16] the authors used neural networks as parametric solution generators for the TSP problem. They provided empirical evidence that optimizing the parameters of the neural network using the policy gradient method results to samplers that generate very good solutions for (Euclidean) TSP instances. Parameterizing the solution generators using neural networks essentially compresses the description of distributions over solutions (the full parameterization would require assigning a parameter to every solution-instance pair ( 𝑠, 𝐼 ) ). Since for most combinatorial problems the size of the solution space is exponentially large (compared to the description of the instance), it is crucial that for such methods to succeed the parameterization must be compressed in the sense that the description of the parameter space 𝒲 is polynomial in the size of the description of the instance family ℐ . Apart from having a tractable number of parameters, it is important that the optimization objective corresponding to the parametric class 𝒫 can provably be optimized using some first-order method in polynomial (in the size of the input) iterations.

We collect these desiderata in the following definition. We denote by [ ℐ ] the description size of ℐ , i.e., the number of bits required to identify any element of ℐ . For instance, if ℐ is the space of unweighted graphs with at most 𝑛 nodes, [ ℐ ] = 𝑂 ( 𝑛 2 ) .

Definition 2 (Complete, Compressed and Efficiently Optimizable Solution Generator) . Fix a prior ℛ over ℐ , a family of solution generators 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} , a loss function ℒ as in Equation (1) and some 𝜖 > 0 .

We say that 𝒫 is complete if there exists some 𝑤 ∈ 𝒲 such that ℒ ( 𝑤 ) ≤ opt + 𝜀 , where opt is defined in (2) .

We say that 𝒫 is compressed if the description size of the parameter space 𝒲 is polynomial in [ ℐ ] and in log(1 /𝜀 ) .

We say that 𝒫 is efficiently optimizable if there exists a first-order method applied on the objective ℒ such that after 𝑇 = poly ([ 𝒲 ] , 1 /𝜀 ) many updates on the parameter vectors, finds an (at most) 𝜖 -sub-optimal vector ̂︀ 𝑤 , i.e., ℒ ( ̂︀ 𝑤 ) ≤ ℒ ( 𝑤 ) + 𝜖 .

Remark 1. We remark that constructing parametric families that are complete and compressed, complete and efficiently optimizable, or compressed and efficiently optimizable (i.e., satisfying any pair of assumptions of Question 1 but not all 3) is usually a much easier task. Consider, for example, the Max-Cut problem on a fixed (unweighted) graph with 𝑛 nodes. Note that ℐ has description size 𝑂 ( 𝑛 2 ) . Solutions of the Max-Cut for a graph with 𝑛 nodes are represented by vertices on the binary hypercube {± 1 } 𝑛 (coordinate 𝑖 dictates the side of the cut that we put node 𝑖 ). One may consider the full parameterization of all distributions over the hypercube. It is not hard to see that this is a complete and efficiently optimizable family (the optimization landscape corresponds to optimizing a linear objective). However, it is not compressed , since it requires 2 𝑛 parameters. On the other extreme, considering a product distribution over coordinates, i.e., we set the value of node 𝑖 to be +1 with probability 𝑝 𝑖 and -1 with 1 -𝑝 𝑖 gives a complete and compressed family. However, as we show in Appendix B, the landscape of this compressed parameterization suffers from highly sub-optimal local minima and therefore, it is not efficiently optimizable .

Therefore, in this work we investigate whether it is possible to have all 3 desiderata of Definition 2 at the same time .

Question 1. Are there complete, compressed, and efficiently optimizable solution generators (i.e., satisfying Definition 2) for challenging combinatorial tasks?

1.2 Our Results

Our Contributions. Before we present our results formally, we summarize the contributions of this work.

Our main contribution is a positive answer (Theorem 1) to Question 1 under general assumptions that capture many combinatorial tasks. We identify a set of conditions (see Assumption 1) that allow us to design a family of solution generators that are complete, compressed and efficiently optimizable.

The conditions are motivated by obstacles that are important for any approach of this nature. This includes solutions that escape to infinity, and also parts of the landscape with vanishing gradient. See the discussion in Section 3 and Figure 1.

We specialize our framework to several important combinatorial problems, some of which are NP-hard, and others tractable: Max-Cut, Min-Cut, Max𝑘 -CSP, Maximum-WeightBipartite-Matching, and the Traveling Salesman Problem.

Finally, we investigate experimentally the effect of the entropy regularizer and the fast/slow mixture scheme that we introduced (see Section 3) and provide evidence that it leads to better solution generators.

We begin with the formal presentation of our assumptions on the feature mappings of the instances and solutions and on the structure of cost function of the combinatorial problem.

Assumption 1 (Structured Feature Mappings) . Let 𝑆 be the solution space and ℐ be the instance space. There exist feature mappings 𝜓 𝑆 : 𝑆 → 𝑋 , for the solutions, and, 𝜓 ℐ : ℐ → 𝑍 , for the instances, where 𝑋,𝑍 are Euclidean vector spaces of dimension 𝑛 𝑋 and 𝑛 𝑍 , such that

(Bounded Feature Spaces) The feature and instance mappings are bounded, i.e., there exist 𝐷 𝑆 , 𝐷 ℐ > 0 such that ‖ 𝜓 𝑆 ( 𝑠 ) ‖ 2 ≤ 𝐷 𝑆 , for all 𝑠 ∈ 𝑆 and ‖ 𝜓 ℐ ( 𝐼 ) ‖ 2 ≤ 𝐷 ℐ , for all 𝐼 ∈ ℐ .

(Bilinear Cost Oracle) The cost of a solution 𝑠 under instance 𝐼 can be expressed as a bilinear function of the corresponding feature vector 𝜓 𝑆 ( 𝑠 ) and instance vector 𝜓 ℐ ( 𝐼 ) , i.e., the solution oracle can be expressed as 𝒪 ( 𝑠, 𝐼 ) = 𝜓 ℐ ( 𝐼 ) ⊤ 𝑀𝜓 𝑆 ( 𝑠 ) for any 𝑠 ∈ 𝑆, 𝐼 ∈ ℐ , for some matrix 𝑀 with ‖ 𝑀 ‖ F ≤ 𝐶 .

(Variance Preserving Features) There exists 𝛼 > 0 such that Var 𝑠 ∼ 𝑈 ( 𝑆 ) [ 𝑣 · 𝜓 𝑆 ( 𝑠 )] ≥ 𝛼 ‖ 𝑣 ‖ 2 2 for any 𝑣 ∈ 𝑋 , where 𝑈 ( 𝑆 ) is the uniform distribution over the solution space 𝑆 .

(Bounded Dimensions/Diameters) The feature dimensions 𝑛 𝑋 , 𝑛 𝑍 , and the diameter bounds 𝐷 𝑆 , 𝐷 ℐ , 𝐶 are bounded above by a polynomial of the description size of the instance space ℐ . The variance lower bound 𝛼 is bounded below by 1 / poly ([ ℐ ]) .

Remark 2 (Boundedness and Bilinear Cost Assumptions) . We remark that Items 1, 4 are simply boundedness assumptions for the corresponding feauture mappings and usually follow easily assuming that we consider reasonable feature mappings. At a high-level, the assumption that the solution is a bilinear function of the solution and instance features (Item 2) prescribes that 'good' feature mappings should enable a simple (i.e., bilinear) expression for the cost function. In the sequel we see that this is satisfied by natural feature mappings for important classes of combinatorial problems.

Remark 3 (Variance Preservation Assumption) . In Item 3 (variance preservation) we require that the solution feature mapping has variance along every direction, i.e., the feature vectors corresponding to the solutions must be 'spread-out' when the underlying solution generator is the uniform distribution. As we show, this assumption is crucial so that the gradients of the resulting optimization objective are not-vanishing, allowing for its efficient optimization.

We mention that various important combinatorial problems satisfy Assumption 1. For instance, Assumption 1 is satisfied by Max-Cut, Min-Cut, Max𝑘 -CSP, Maximum-Weight-Bipartite-Matching and Traveling Salesman. We refer the reader to the upcoming Section 2 for an explicit description of the structured feature mappings for these problems. Having discussed Assumption 1, we are ready to state our main abstract result which resolves Question 1.

Theorem 1. Consider a combinatorial problem with instance space ℐ that satisfies Assumption 1. For any prior ℛ over ℐ and 𝜖 > 0 , there exists a family of solution generators 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} with parameter space 𝒲 that is complete, compressed and, efficiently optimizable.

A sketch behind the design of the family 𝒫 can be found in Section 3 and Section 4.

Remark 4 (Computational Barriers in Sampling) . We note that the families of generative models (a.k.a., solution generators) that we provide have polynomial parameter complexity and are optimizable in a small number of steps using gradient-based methods. Hence, in a small number of iterations, gradient-based methods converge to distributions whose mass is concentrated on nearly optimal solutions. This holds, as we show, even for challenging ( NP -hard) combinatorial problems. Our results do not, however, prove P = NP , as it may be computationally hard to sample from our generative models. We remark that while such approaches are in theory hard, such models seem to perform remarkably well experimentally where sampling is based on Langevin dynamics techniques [SE20, SSDK + 20]. Though as our theory predicts, and simulations support, landscape problems seem to be a direct impediment even to obtain good approximate solutions.

Remark 5 (Neural Networks as Solution Samplers) . A natural question would be whether our results can be extended to the case where neural networks are (efficient) solution samplers, as in [BPL + 16]. Unfortunately, a benign landscape result for neural network solution generators most likely cannot exist. It is well-known that end-to-end theoretical guarantees for training neural networks are out of reach since the corresponding optimization tasks are provably computationally intractable, see, e.g., [CGKM22] and the references therein.

Finally, we would like to mention an interesting aspect of Assumption 1. Given a combinatorial problem, Assumption 1 essentially asks for the design of feature mappings for the solutions and the instances that satisfy desiderata such as boundedness and variance preservation. Max-Cut, Min-Cut, TSP and Max𝑘 -CSP and other problems satisfy Assumption 1 because we managed to design appropriate (problem-specific) feature mappings that satisfy the requirements of Assumption 1. There are interesting combinatorial problems for which we do not know how to design such good feature mappings. For instance, the "natural" feature mapping for the Satisfiability problem (SAT) (similar to the one we used for Max𝑘 -CSPs) would require feature dimension exponential in the size of the instance (we need all possible monomials of 𝑛 variables and degree at most 𝑛 ) and therefore, would violate Item 4 of Assumption 1.

1.3 Related Work

Neural Combinatorial Optimization. Tackling combinatorial optimization problems constitutes one of the most fundamental tasks of theoretical computer science [GLS12, PS98, S + 03, Sch05, CLS + 95] and various approaches have been studied for these problems such as local search methods, branch-and-bound algorithms and meta-heuristics such as genetic algorithms and simulated annealing. Starting from the seminal work of [HT85], researchers apply neural networks [Smi99, VFJ15, BPL + 16] to solve combinatorial optimization tasks. In particular, researchers have explored the power of machine learning, reinforcement learning and deep learning methods for solving combinatorial optimization problems [BPL + 16, YW20, LZ09, DCL + 18, BLP21, MSIB21, NOST18, SHM + 16, MKS + 13, SSS + 17, ER18, KVHW18, ZCH + 20, CCK + 21, MGH + 19, GCF + 19, KLMS19].

The use of neural networks in combinatorial problems is extensive [SLB + 18, JLB19, GCF + 19, YGS20, MSIB21, BPL + 16, KDZ + 17, YP19, CT19, YBV19, KCK + 20, KCY + 21, DAT20, NJS + 20, TRWG21, AMW18, KL20, Jeg22, SBK22, ART23] and various papers aim to understand the theoretical ability of neural networks to solve such problems [HS23b, HS23a, Gam23]. Our paper builds on the framework of the influential experimental work of [BPL + 16] to tackle combinatorial optimization problems such as TSP using neural networks and reinforcement learning. [KP + 21] uses an entropy maximization scheme in order to generate diversified candidate solutions. This experimental heuristic is quite close to our theoretical idea for entropy regularization. In our work, entropy regularization allows us to design quasar-convex landscapes and the fast/slow mixing scheme to obtain diversification of solutions. Among other related applied works, [KCK + 20, KPP22] study the use of Transformer architectures combined with the Reinforce algorithm employing symmetries (i.e., the existence of multiple optimal solutions of a CO problem) improving the generalization capability of Deep RL NCO and [MLC + 21] studies Transformer architectures and aims to learn improvement heuristics for routing problems using RL.

Gradient Descent Dynamics. Our work provides theoretical understanding on the gradient-descent landscape arising in NCO problems. Similar questions regarding the dynamics of gradient descent have been studied in prior work concerning neural networks; for instance, [AS20] and [AKM + 21] fix the algorithm (SGD on neural networks) and aim to understand the power of this approach (which function classes can be learned). Various other works study gradient descent dynamics in neural networks. We refer to [AS18, AS20, ABAB + 21, MYSSS21, BEG + 22, DLS22, ABA22, AAM22, BBSS22, ABAM23, AKM + 21, EGK + 23] (and the references therein) for a small sample of this line of research.

2 Combinatorial Applications

We now consider concrete combinatorial problems and show that there exist appropriate and natural feature mappings for the solutions and instances that satisfy Assumption 1; so Theorem 1 is applicable for any such combinatorial task. For a more detailed treatment, we refer to Appendix G.

Min-Cut and Max-Cut. Min-Cut (resp. Max-Cut) are central graph combinatorial problems where the task is to split the nodes of the graph in two subsets so that the number of edges from one subset to the other (edges of the cut) is minimized (resp. maximized). Given a graph 𝐺 with 𝑛 nodes represented by its Laplacian matrix 𝐿 𝐺 = 𝐷 -𝐴 , where 𝐷 is the diagonal degree matrix and 𝐴 is the adjacency matrix of the graph, the goal in the Min-Cut (resp. Max-Cut) problem is to find a solution vector 𝑠 ∈ {± 1 } 𝑛 so that 𝑠 ⊤ 𝐿 𝐺 𝑠/ 4 is minimized (resp. maximized).

We first show that there exist natural feature mappings so that the cost of every solution 𝑠 under any instance/graph 𝐺 is a bilinear function of the feature vectors, see Item 2 of Assumption 1. We consider the correlation-based feature mapping 𝜓 𝑆 ( 𝑠 ) = ( 𝑠𝑠 ⊤ ) ♭ ∈ R 𝑛 2 , where by ( · ) ♭ we denote the vectorization/flattening operation and the Laplacian for the instance (graph), 𝜓 ℐ ( 𝐺 ) = ( 𝐿 𝐺 ) ♭ ∈ R 𝑛 2 . Then simply setting the matrix 𝑀 to be the identity 𝐼 ∈ R 𝑛 2 × 𝑛 2 the cost of any solution 𝑠 can be expressed as the bilinear function 𝜓 ℐ ( 𝐺 ) ⊤ 𝑀𝜓 𝑆 ( 𝑠 ) = ( 𝐿 ♭ 𝐺 ) ⊤ ( 𝑠𝑠 ⊤ ) ♭ = 𝑠 ⊤ 𝐿 𝐺 𝑠 . We observe that for (unweighted) graphs with 𝑛 nodes the description size of the family of all instances ℐ is roughly 𝑂 ( 𝑛 2 ) , and therefore the dimensions of the feature mappings 𝜓 𝑆 , 𝜓 ℐ are clearly polynomial in the description size of ℐ . Moreover, considering unweighted graphs, it holds that ‖ 𝜓 ℐ ( 𝐺 ) ‖ 2 , ‖ 𝜓 𝑆 ( 𝑠 ) ‖ 2 , ‖ 𝑀 ‖ F ≤ poly ( 𝑛 ) . Therefore, the constants 𝐷 𝑆 , 𝐷 ℐ , 𝐶 are polynomial in the description size of the instance family.

̸

It remains to show that our solution feature mapping satisfies the variance preservation assumption, i.e., Item 3 in Assumption 1. A uniformly random solution vector 𝑠 ∈ {± 1 } 𝑛 is sampled by setting each 𝑠 𝑖 = 1 with probability 1 / 2 independently. In that case, we have E [ 𝑣 · 𝑥 ] = 0 and therefore Var ( 𝑣 · 𝑥 ) = E [( 𝑣 · 𝑥 ) 2 ] = ∑︀ 𝑖 = 𝑗 𝑣 𝑖 𝑣 𝑗 E [ 𝑥 𝑖 𝑥 𝑗 ] = ∑︀ 𝑖 𝑣 2 𝑖 = ‖ 𝑣 ‖ 2 2 , since, by the independence of 𝑥 𝑖 , 𝑥 𝑗 , the cross-terms of the sum vanish. We observe that the same hold true for the Max-Cut problem and therefore, structured feature mappings exist for Max-Cut as well (where 𝐿 ( 𝑠 ; 𝐺 ) = -𝑠 ⊤ 𝐿 𝐺 𝑠 ) . We shortly mention that there also exist structured feature mappings for Max𝑘 -CSP. We refer to Theorem 4 for further details.

Remark 6 (Partial Instance Information/Instance Context) . We remark that Assumption 1 allows for the 'instance' 𝐼 to only contain partial information about the actual cost function. For example, consider the setting where each sampled instance is an unweighted graph 𝐺 but the cost oracle takes the form 𝒪 ( 𝐺,𝑠 ) = ( 𝐿 𝐺 ) ♭ 𝑀 ( 𝑠𝑠 ⊤ ) ♭ for a matrix 𝑀 𝑖𝑗 = 𝑎 𝑖 when 𝑖 = 𝑗 and 𝑀 𝑖𝑗 = 0 otherwise. This cost function models having a unknown weight function , i.e., the weight of edge 𝑖 of 𝐺 is 𝑎 𝑖 if edge 𝑖 exists in the observed instance 𝐺 , on the edges of the observed unweighted graph 𝐺 , that the algorithm has to learn in order to be able to find the minimum or maximum cut. For simplicity, in what follows, we will continue referring to 𝐼 as the instance even though it may only contain partial information about the cost function of the underlying combinatorial problem.

Maximum-Weight-Bipartite-Matching and TSP. The Maximum-Weight-Bipartite-Matching (MWBP) problem is another graph problem that, given a bipartite graph 𝐺 with 𝑛 nodes and 𝑚 edges, asks for the maximum-weight matching. The feature vector corresponding to a matching can be represented as a binary matrix 𝑅 ∈ { 0 , 1 } 𝑛 × 𝑛 with ∑︀ 𝑗 𝑅 𝑖𝑗 = 1 for all 𝑖 and ∑︀ 𝑖 𝑅 𝑖𝑗 = 1 for all 𝑗 , i.e., 𝑅 is a permutation matrix. Therefore, for a candidate matching 𝑠 , we set 𝜓 𝑆 ( 𝑠 ) to be the matrix 𝑅 defined above. Moreover, the feature vector of the graph is the (negative flattened) adjacency matrix 𝐸 ♭ . The cost oracle is then 𝐿 ( 𝑅 ; 𝐸 ) = ∑︀ 𝑖𝑗 𝐸 𝑖𝑗 𝑀 𝑖𝑗 𝑅 𝑖𝑗 perhaps for an unknown weight matrix 𝑀 𝑖𝑗 (see Remark 6). For the Traveling Salesman Problem (TSP) the feature vector is again a matrix 𝑅 with the additional constraint that 𝑅 has to represent a single cycle (a tour over all cities). The cost function for TSP is again 𝐿 ( 𝑅 ; 𝐸 ) = ∑︀ 𝑖𝑗 𝐸 𝑖𝑗 𝑀 𝑖𝑗 𝑅 𝑖𝑗 . One can check that those representations of the instance and solution satisfy the assumptions of Items 1 and 4. Showing that the variance of those representations has a polynomial lower bound is more subtle and we refer the reader to the Supplementary Material.

We shortly mention that the solution generators for Min-Cut and Maximum-Weight-BipartiteMatching are also efficiently samplable.

3 Optimization Landscape

Exponential Families as Solution Generators. A natural candidate to construct our family of solution generators is to consider the distribution that assigns to each solution 𝑠 ∈ 𝑆 and instance 𝐼 ∈ ℐ mass proportional to its score exp( -𝜏𝐿 ( 𝑠 ; 𝐼 )) = exp( -𝜏𝜓 ℐ ( 𝐼 ) ⊤ 𝑀𝜓 𝑆 ( 𝑠 )) = exp( -𝜏𝑧 ⊤ 𝑀𝑥 )

5

-5

0

6

-5

16

-14

8

10

-10

18

-5

12

20

-5

5

5

for some 'temperature' parameter 𝜏 , where 𝜓 ℐ and 𝜓 𝑆 are the feature mappings promised to exist due to Assumption 1, 𝑧 = 𝜓 ℐ ( 𝐼 ) , and, 𝑥 = 𝜓 𝑆 ( 𝑠 ) . Note that as long as 𝜏 → + ∞ , this distribution tends to concentrate on solutions that achieve small loss.

Remark 7. To construct the above solution sampler one could artificially query specific solutions to the cost oracle of Definition 1 and try to learn the cost matrix 𝑀 . However, we remark that our goal (see Definition 2) is to show that we can train a parametric family via gradient-based methods so that it generates (approximately) optimal solutions and not to simply learn the cost matrix 𝑀 via some other method and then use it to generate good solutions.

Obstacle I: Minimizers at Infinity. One could naturally consider the parametric family 𝜑 ( 𝑥 ; 𝑧 ; 𝑊 ) ∝ exp( 𝑧 ⊤ 𝑊𝑥 ) (note that with 𝑊 = -𝜏𝑀 , we recover the distribution of the previous paragraph) and try to perform gradient-based methods on the loss (recall that 𝐿 ( 𝑥 ; 𝑧 ) = 𝑧 ⊤ 𝑀𝑥 ) 1

The question is whether gradient updates on the parameter 𝑊 eventually converge to a matrix 𝑊 whose associated distribution 𝜑 ( 𝑊 ) generates near-optimal solutions (note that the matrix -𝜏𝑀 with 𝜏 → + ∞ is such a solution). After computing the gradient of ℒ , we observe that

where the inner product between two matrices 𝐴 · 𝐵 is the trace Tr( 𝐴 ⊤ 𝐵 ) = ∑︀ 𝑖,𝑗 𝐴 𝑖𝑗 𝐵 𝑖𝑗 . This means that the gradient field of ℒ always has a contribution to the direction of 𝑀 . Nevertheless the actual minimizer is at infinity, i.e., it corresponds to the point 𝑊 = -𝜏𝑀 when 𝜏 → + ∞ . While the correlation with the optimal point is positive (which is encouraging), having such contribution to this direction is not a sufficient condition for actually reaching 𝑊 . The objective has vanishing gradients at infinity and gradient descent may get trapped in sub-optimal stationary points, see the left plot in Figure 1.

Solution I: Quasar Convexity via Entropy Regularization. Our plan is to try and make the objective landscape more benign by adding an entropy-regularizer. Instead of trying to make the objective convex (which may be too much to ask in the first place) we are able obtain a much better landscape with a finite global minimizer and a gradient field that guides gradient descent to the minimizer. Those properties are described by the so-called class of 'quasar-convex' functions. Quasar convexity (or weak quasi-convexity [HMR16]) is a well-studied notion in optimization [HMR16, HSS20, LV16, ZMB + 17, HLSS15] and can be considered as a high-dimensional generalization of unimodality.

Definition 3 (Quasar Convexity [HMR16, HSS20]) . Let 𝛾 ∈ (0 , 1] and let 𝑥 be a minimizer of the differentiable function 𝑓 : R 𝑛 → R . The function 𝑓 is 𝛾 -quasar-convex with respect to 𝑥 on a domain 𝐷 ⊆ R 𝑛 if for all 𝑥 ∈ 𝐷 , ∇ 𝑓 ( 𝑥 ) · ( 𝑥 -𝑥 ) ≥ 𝛾 ( 𝑓 ( 𝑥 ) -𝑓 ( 𝑥 )) .

In the above definition, notice that the main property that we need to establish is that the gradient field of our objective correlates positively with the direction 𝑊 -𝑊 , where 𝑊 is its minimizer. We denote by 𝐻 : 𝒲 → R the negative entropy of 𝜑 ( 𝑊 ) , i.e.,

and consider the regularized objective

for some 𝜆 > 0 . We show (follows from Lemma 4) that the gradient-field of the regularized objective indeed 'points' towards a finite minimizer (the matrix 𝑊 = -𝑀/𝜆 ):

where the randomness is over 𝑧 ∼ ℛ , 𝑥 ∼ 𝜑 ( · ; 𝑧 ; 𝑊 ) . Observe that now the minimizer of ℒ 𝜆 is the point -𝑀/𝜆 , which for 𝜆 = poly ( 𝜖, 𝛼, 1 /𝐶, 1 /𝐷 𝑆 , 1 /𝐷 ℐ ) (these are the parameters of Assumption 1) is promised to yield a solution sampler that generates 𝜖 -sub-optimal solutions (see also Proposition 2 and Appendix C). Having the property of Equation (4) suffices for showing that a gradient descent iteration (with an appropriately small step-size) will eventually converge to the minimizer.

Obstacle II: Vanishing Gradients. While we have established that the gradient field of the regularized objective 'points' towards the right direction, the regularized objective still suffers from vanishing gradients, see the middle plot in Figure 1. In other words, 𝛾 in the definition of quasar convexity (Definition 3) may be exponentially small, as it is proportional to the variance of the random variable 𝑧 ⊤ ( 𝑊 + 𝑀/𝜆 ) 𝑥 , see Equation (4). As we see in the middle plot of Figure 1, the main issue is the vanishing gradient when 𝑊 gets closer to the minimizer -𝑀/𝜆 (towards the front-corner). For simplicity, consider the variance along the direction of 𝑊 , i.e., Var [ 𝑧 ⊤ 𝑊𝑥 ] and recall that 𝑥 is generated by the density exp( 𝑧 ⊤ 𝑊𝑥 ) / ( ∑︀ 𝑥 ∈ 𝑋 exp( 𝑧 ⊤ 𝑊𝑥 )) . When ‖ 𝑊 ‖ 2 → + ∞ we observe that the value 𝑧 ⊤ 𝑊𝑥 concentrates exponentially fast to max 𝑥 ∈ 𝑋 𝑧 ⊤ 𝑊𝑥 (think of the convergence of the soft-max to the max function). Therefore, the variance Var [ 𝑧 ⊤ 𝑊𝑥 ] may vanish exponentially fast making the convergence of gradient descent slow.

Solution II: Non-Vanishing Gradients via Fast/Slow Mixture Generators. We propose a fix to the vanishing gradients issue by using a mixture of exponential families as a solution generator. We define the family of solution generators 𝒫 = { 𝑝 ( 𝑊 ) : 𝑊 ∈ 𝒲} to be

for a (fixed) mixing parameter 𝛽 ⋆ and a (fixed) temperature parameter 𝜌 ⋆ . The main idea is to have the first component of the mixture to converge fast to the optimal solution (to -𝑀/𝜆 ) while the second 'slow' component that has parameter 𝜌 ⋆ 𝑊 stays closer to the uniform distribution over solutions that guarantees non-trivial variance (and therefore non-vanishing gradients).

More precisely, taking 𝜌 ⋆ to be sufficiently small, the distribution 𝜑 ( 𝜌 ⋆ 𝑊 ) is almost uniform over the solution space 𝜓 𝑆 ( 𝑆 ) . Therefore, in Equation (4), the almost uniform distribution component of the mixture will add to the variance and allow us to show a lower bound. This is where Item 3 of Assumption 1 comes into play and gives us the desired non-trivial variance lower bound under the uniform distribution. We view this fast/slow mixture technique as an interesting insight of our work: we use the 'fast' component (the one with parameter 𝑊 ) to actually reach the optimal solution -𝑀/𝜆 and and we use the 'slow' component (the one with parameter 𝜌 ⋆ 𝑊 that essentially generates random solutions) to preserve a non-trivial variance lower bound during optimization.

4 Complete, Compressed and Efficiently Optimizable Solution Generators

In this section, we discuss the main results that imply Theorem 1: the family 𝒫 of Equation (5) is complete, compressed and efficiently optimizable (for some choice of 𝛽 ⋆ , 𝜌 ⋆ and 𝒲 ).

Completeness. First, we show that the family of solution generators of Equation (5) is complete. For the proof, we refer to Proposition 2 in Appendix C. At a high-level, we to pick 𝛽 ⋆ , 𝜌 ⋆ to be of order poly ( 𝜖, 𝛼, 1 /𝐶, 1 /𝐷 𝑆 , 1 /𝐷 ℐ ) . This yields that the matrix 𝑊 = -𝑀/𝜆 is such that ℒ ( 𝑊 ) ≤ opt+ 𝜖 , where 𝑀 is the matrix of Item 2 in Assumption 1 and 𝜆 is poly ( 𝜖/ [ ℐ ]) . To give some intuition about this choice of matrix, let us see how ℒ ( 𝑊 ) behaves. By definition, we have that

where the distribution 𝑝 belongs to the family of Equation (5), i.e., 𝑝 ( 𝑊 ) = (1 -𝛽 ⋆ ) 𝜑 ( 𝑊 ) + 𝛽 ⋆ 𝜑 ( 𝜌 ⋆ 𝑊 ) . Since the mixing weight 𝛽 ⋆ is small, we have that 𝑝 ( 𝑊 ) is approximately equal to 𝜑 ( 𝑊 ) . This means that our solution generator draws samples from the distribution whose mass at 𝑥 given instance 𝑧 is proportional to exp( -𝑧 ⊤ 𝑀𝑥/𝜆 ) and, since 𝜆 > 0 is very small, the distribution concentrates to solutions 𝑥 that tend to minimize the objective 𝑧 ⊤ 𝑀𝑥 . This is the reason why 𝑊 = -𝑀/𝜆 is close to opt in the sense that ℒ ( 𝑊 ) ≤ opt + 𝜖 .

Compression. As a second step, we show (in Proposition 3, see Appendix D) that 𝒫 is a compressed family of solution generators. This result follows immediately from the structure of Equation (5) (observe that 𝑊 has 𝑛 𝑋 𝑛 𝑍 parameters) and the boundedness of 𝑊 = -𝑀/𝜆 .

Efficiently Optimizable. The proof of this result essentially corresponds to the discussion provided in Section 3. Our main structural result shows that the landscape of the regularized objective with the fast/slow mixture solution-generator is quasar convex. More precisely, we consider the following objective:

where 𝑝 ( 𝑊 ) belongs in the family 𝒫 of Equation (5) and 𝑅 is a weighted sum of two negative entropy regularizers (to be in accordance with the mixture structure of 𝒫 ) , i.e., 𝑅 ( 𝑊 ) = (1 -𝛽 ⋆ ) 𝐻 ( 𝑊 ) + 𝛽 ⋆ /𝜌 ⋆ 𝐻 ( 𝜌 ⋆ 𝑊 ) . Our main structural results follows (for the proof, see Appendix E.1).

Proposition 1 (Quasar Convexity) . Consider 𝜖 > 0 and a prior ℛ over ℐ . Assume that Assumption 1 holds. The function ℒ 𝜆 of Equation (1) with domain 𝒲 is poly ( 𝜖, 𝛼, 1 /𝐶, 1 /𝐷 𝑆 , 1 /𝐷 ℐ ) -quasar convex with respect to -𝑀/𝜆 on the domain 𝒲 .

Since 𝜌 ⋆ is small (by Proposition 2), 𝐻 ( 𝜌 ⋆ 𝑊 ) is essentially constant and close in value to the negative entropy of the uniform distribution. Hence, the effect of 𝑅 ( 𝑊 ) during optimization is essentially the same as that of 𝐻 ( 𝑊 ) (since 𝛽 ⋆ is close to 0). We show that ℒ 𝜆 is quasar convex with a non-trivial parameter 𝛾 (see Proposition 1). We can then apply (in a black-box manner) the convergence results from [HMR16] to optimize it using projected SGD. We show that SGD finds a weight matrix ̂︁ 𝑊 such that the solution generator 𝑝 ( ̂︁ 𝑊 ) generates solutions achieving actual loss ℒ close to that of the near optimal matrix 𝑊 = -𝑀/𝜆 , i.e., ℒ ( ̂︁ 𝑊 ) ≤ ℒ ( 𝑊 ) + 𝜖 . For further details, see Appendix E.3.

5 Experimental Evaluation

In this section, we investigate experimentally the effect of our main theoretical contributions, the entropy regularizer (see Equation (2)) and the fast/slow mixture scheme (see Equation (5)). We try to find the Max-Cut of a fixed graph 𝐺 , i.e., the support of the prior ℛ is a single graph. Similarly to our theoretical results, our sampler is of the form 𝑒 score( 𝑠 ; 𝑤 ) , where 𝑠 ∈ {-1 , 1 } 𝑛 (here 𝑛 is the number of nodes in the graph) is a candidate solution of the Max-Cut problem. For the score function we used a simple linear layer (left plot of Figure 2) and a 3-layer ReLU network (right plot of Figure 2).

Small Graph Instances. Focusing on instances where the number of nodes 𝑛 is small (say 𝑛 = 15 ), we can explicitly compute the density function and work with an exact sampler. We generate 100 random 𝐺 ( 𝑛, 𝑝 ) graphs with 𝑛 = 15 nodes and 𝑝 = 0 . 5 and train solution generators using both the 'vanilla' loss ℒ and the entropy-regularized loss ℒ 𝜆 with the fast/slow mixture scheme. We perform 600 iterations and, for the entropy regularization, we progressively decrease the regularization weight, starting from 10, and dividing it by 2 every 60 iterations. Out of the 100 trials we found that our proposed objective was always able to find the optimal cut while the model trained with the vanilla loss was able to find it for approximately 65% of the graphs (for 65 out of 100 using the linear network and for 66 using the ReLU network).

Hence, our experiments demonstrate that while the unregularized objective is often 'stuck' at suboptimal solutions - and this happens even for very small instances ( 𝑛 = 15 nodes) - of the Max-Cut problem, the objective motivated by our theoretical results is able to find the optimal solutions. For further details, see Appendix I.

Regularized Loss

8

Regularized Loss

42

36

40

34

38

十

32

36

30

34

28

32

0

100

200

0OE

400

500

600

0

100

200

400

500

600

Large Graph Instances. A natural question is whether this improvement is also apparent in larger graphs. We focus on the case of random 𝑑 -regular graphs with 𝑛 nodes. It is well-known that for this family of graphs, with high probability as 𝑛 → ∞ , the size of the maximum cut satisfies MaxCut( 𝐺 ( 𝑛, 𝑑 )) = 𝑛 ( 𝑑/ 4 + 𝑃 ⋆ √︀ 𝑑/ 4 + 𝑜 𝑑 ( √ 𝑑 )) + 𝑜 ( 𝑛 ) , where 𝑃 ⋆ ≈ 0 . 7632 is a universal constant [DMS17]. We aim to find a good approximation for the normalized cut-value, defined as (cut _ value /𝑛 -𝑑/ 4) / √︀ 𝑑/ 4 , which (roughly speaking) takes values in [0 , 𝑃 ⋆ ] . We obtain approximate random samples from the density 𝑒 𝑓 using the Metropolis-Adjusted Langevin Algorithm (MALA). In particular, an approximate sample from this density is obtained by the Euler-Maruyama method for simulating the Langevin diffusion: 𝑥 𝑘 +1 = 𝑥 𝑘 + 𝜏 ∇ 𝑓 ( 𝑥 𝑘 ) + √ 2 𝜏𝜉 𝑘 , where 𝜉 𝑘 is an independent Gaussian vector 𝒩 (0 , 𝐼 ) . MALA incorporates an additional step based on the Metropolis-Hastings algorithm (see [Bes94, SK21]). In our case, the score function 𝑓 is a simple 3-layer ReLU network. In our experiments for 3 larger random regular graphs (600 nodes) using the fast/slow mixing technique along entropy regularization we see that our method leads to improvements over the vanilla objective. Plots of the trajectories of the vanilla and our method can be found in Figure 3. In the horizontal axis we plot the iterations and in the vertical axis we plot the normalized cut score of each method (higher is better) - we stop the plot of the vanilla trajectory after 200 iterations because we observed that its output has fully converged and is stuck.

0.200

0.175

0.150

0.175

0.150

0.125

0.150

0.125

0.100

0.125

0.100

0.075

0.100

0.075

0.075

0.050

0.050

0.050

0.025

0.000

0.025

0

200

400

600

Regularized Loss

800

1000

0.025

0.000

0

200

400

600

Regularized Loss

800

1000

0.000

0

200

400

600

Regularized Loss

800

1000

References

[AAM22] Emmanuel Abbe, Enric Boix Adsera, and Theodor Misiakiewicz. The merged-staircase property: a necessary and nearly sufficient condition for sgd learning of sparse functions on two-layer neural networks. In Conference on Learning Theory , pages 4782-4887. PMLR, 2022. 5

[ABA22] Emmanuel Abbe and Enric Boix-Adsera. On the non-universality of deep learning: quantifying the cost of symmetry. arXiv preprint arXiv:2208.03113 , 2022. 5

[ABAB + 21] Emmanuel Abbe, Enric Boix-Adsera, Matthew S Brennan, Guy Bresler, and Dheeraj Nagaraj. The staircase property: How hierarchical structure can guide deep learning. Advances in Neural Information Processing Systems , 34:26989-27002, 2021. 5

[ABAM23] Emmanuel Abbe, Enric Boix-Adsera, and Theodor Misiakiewicz. Sgd learning on neural networks: leap complexity and saddle-to-saddle dynamics. arXiv preprint arXiv:2302.11055 , 2023. 5

[AEG + 22] Simon Apers, Yuval Efron, Pawel Gawrychowski, Troy Lee, Sagnik Mukhopadhyay, and Danupon Nanongkai. Cut query algorithms with star contraction. arXiv preprint arXiv:2201.05674 , 2022. 2

[AKM + 21] Emmanuel Abbe, Pritish Kamath, Eran Malach, Colin Sandon, and Nathan Srebro. On the power of differentiable learning versus pac and sq learning. Advances in Neural Information Processing Systems , 34:24340-24351, 2021. 5

[AMW18] Saeed Amizadeh, Sergiy Matusevych, and Markus Weimer. Learning to solve circuitsat: An unsupervised differentiable approach. In International Conference on Learning Representations , 2018. 5

[ART23] Maria Chiara Angelini and Federico Ricci-Tersenghi. Modern graph neural networks do worse than classical greedy algorithms in solving combinatorial optimization problems like maximum independent set. Nature Machine Intelligence , 5(1):29-31, 2023. 5

[AS18] Emmanuel Abbe and Colin Sandon. Provable limitations of deep learning. arXiv preprint arXiv:1812.06369 , 2018. 5

[AS20] Emmanuel Abbe and Colin Sandon. On the universality of deep learning. Advances in Neural Information Processing Systems , 33:20061-20072, 2020. 5

[BBSS22] Alberto Bietti, Joan Bruna, Clayton Sanford, and Min Jae Song. Learning single-index models with shallow neural networks. Advances in Neural Information Processing Systems , 35:9768-9783, 2022. 5

[BEG + 22] Boaz Barak, Benjamin L Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang. Hidden progress in deep learning: Sgd learns parities near the computational limit. arXiv preprint arXiv:2207.08799 , 2022. 5

[Bes94] Julian Besag. Comments on 'representations of knowledge in complex systems' by u. grenander and mi miller. J. Roy. Statist. Soc. Ser. B , 56(591-592):4, 1994. 10

[BLP21] Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimization: a methodological tour d'horizon. European Journal of Operational Research , 290(2):405-421, 2021. 5

[BPL + 16] Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940 , 2016. 1, 2, 3, 5

[CCK + 21] Quentin Cappart, Didier Chételat, Elias Khalil, Andrea Lodi, Christopher Morris, and Petar Veliˇ ckovi´ c. Combinatorial optimization and reasoning with graph neural networks. arXiv preprint arXiv:2102.09544 , 2021. 5

[CGG + 19] Zongchen Chen, Andreas Galanis, Leslie Ann Goldberg, Will Perkins, James Stewart, and Eric Vigoda. Fast algorithms at low temperatures via markov chains. arXiv preprint arXiv:1901.06653 , 2019. 27

[CGKM22] Sitan Chen, Aravind Gollakota, Adam Klivans, and Raghu Meka. Hardness of noisefree learning for two-hidden-layer neural networks. Advances in Neural Information Processing Systems , 35:10709-10724, 2022. 5

[CLS + 95] William Cook, László Lovász, Paul D Seymour, et al. Combinatorial optimization: papers from the DIMACS Special Year , volume 20. American Mathematical Soc., 1995. 1, 5

[CLV22] Zongchen Chen, Kuikui Liu, and Eric Vigoda. Spectral independence via stability and applications to holant-type problems. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS) , pages 149-160. IEEE, 2022. 27

[COLMS22] Amin Coja-Oghlan, Philipp Loick, Balázs F Mezei, and Gregory B Sorkin. The ising antiferromagnet and max cut on random regular graphs. SIAM Journal on Discrete Mathematics , 36(2):1306-1342, 2022. 27

[CT19] Xinyun Chen and Yuandong Tian. Learning to perform local rewriting for combinatorial optimization. Advances in Neural Information Processing Systems , 32, 2019. 5

[CZ22] Xiaoyu Chen and Xinyuan Zhang. A near-linear time sampler for the ising model. arXiv preprint arXiv:2207.09391 , 2022. 27

[d'A08] Alexandre d'Aspremont. Smooth optimization with approximate gradient. SIAM Journal on Optimization , 19(3):1171-1183, 2008. 28

[DAT20] Arthur Delarue, Ross Anderson, and Christian Tjandraatmadja. Reinforcement learning with combinatorial actions: An application to vehicle routing. Advances in Neural Information Processing Systems , 33:609-620, 2020. 5

[DCL + 18] Michel Deudon, Pierre Cournut, Alexandre Lacoste, Yossiri Adulyasak, and LouisMartin Rousseau. Learning heuristics for the tsp by policy gradient. In International conference on the integration of constraint programming, artificial intelligence, and operations research , pages 170-181. Springer, 2018. 5

[DLS22] Alexandru Damian, Jason Lee, and Mahdi Soltanolkotabi. Neural networks can learn representations with gradient descent. In Conference on Learning Theory , pages 5413-5452. PMLR, 2022. 5

[DMS17] Amir Dembo, Andrea Montanari, and Subhabrata Sen. Extremal cuts of sparse random graphs. The Annals of Probability , 45(2):1190-1217, 2017. 10

[dPS97] JC Anglès d'Auriac, M Preissmann, and A Sebö. Optimal cuts in graphs and statistical mechanics. Mathematical and Computer Modelling , 26(8-10):1-11, 1997. 27

[Edm65] Jack Edmonds. Maximum matching and a polyhedron with 0, 1-vertices. Journal of research of the National Bureau of Standards B , 69(125-130):55-56, 1965. 29

[EGK + 23] Benjamin L Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang. Pareto frontiers in neural feature learning: Data, compute, width, and luck. arXiv preprint arXiv:2309.03800 , 2023. 5

[ER18] Patrick Emami and Sanjay Ranka. Learning permutations with sinkhorn policy gradient. arXiv preprint arXiv:1805.07010 , 2018. 5

[Gam23] David Gamarnik. Barriers for the performance of graph neural networks (gnn) in discrete random structures. a comment on ∖ cite { schuetz2022combinatorial } , ∖ cite { angelini2023modern } , ∖ cite { schuetz2023reply } . arXiv preprint arXiv:2306.02555 , 2023. 5

[GCF + 19] Maxime Gasse, Didier Chételat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. Exact combinatorial optimization with graph convolutional neural networks. Advances in Neural Information Processing Systems , 32, 2019. 5

[GLS12] Martin Grötschel, László Lovász, and Alexander Schrijver. Geometric algorithms and combinatorial optimization , volume 2. Springer Science & Business Media, 2012. 1, 5

[GPRW19] Andrei Graur, Tristan Pollner, Vidhya Ramaswamy, and S Matthew Weinberg. New query lower bounds for submodular function minimization. arXiv preprint arXiv:1911.06889 , 2019. 2

[HLSS15] Elad Hazan, Kfir Levy, and Shai Shalev-Shwartz. Beyond convexity: Stochastic quasiconvex optimization. Advances in neural information processing systems , 28, 2015. 7

[HMR16] Moritz Hardt, Tengyu Ma, and Benjamin Recht. Gradient descent learns linear dynamical systems. arXiv preprint arXiv:1609.05191 , 2016. 7, 8, 9, 17, 23, 24

[HS23a] Christoph Hertrich and Leon Sering. Relu neural networks of polynomial size for exact maximum flow computation. In International Conference on Integer Programming and Combinatorial Optimization , pages 187-202. Springer, 2023. 5

[HS23b] Christoph Hertrich and Martin Skutella. Provably good solutions to the knapsack problem via neural networks of bounded size. INFORMS Journal on Computing , 2023. 5

[HSS20] Oliver Hinder, Aaron Sidford, and Nimit Sohoni. Near-optimal methods for minimizing star-convex functions and beyond. In Conference on learning theory , pages 1894-1938. PMLR, 2020. 7, 8

[HT85] John J Hopfield and David W Tank. 'neural' computation of decisions in optimization problems. Biological cybernetics , 52(3):141-152, 1985. 1, 5

[Jeg22] Stefanie Jegelka. Theory of graph neural networks: Representation and learning. arXiv preprint arXiv:2204.07697 , 2022. 5

[Jer03] Mark Jerrum. Counting, sampling and integrating: algorithms and complexity . Springer Science & Business Media, 2003. 30

[JLB19] Chaitanya K Joshi, Thomas Laurent, and Xavier Bresson. An efficient graph convolutional network technique for the travelling salesman problem. arXiv preprint arXiv:1906.01227 , 2019. 5

[JS93] Mark Jerrum and Alistair Sinclair. Polynomial-time approximation algorithms for the ising model. SIAM Journal on computing , 22(5):1087-1116, 1993. 27

[JSV04] Mark Jerrum, Alistair Sinclair, and Eric Vigoda. A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries. Journal of the ACM (JACM) , 51(4):671-697, 2004. 30

[Kak01] Sham M Kakade. A natural policy gradient. Advances in neural information processing systems , 14, 2001. 2

[KCK + 20] Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, Iljoo Yoon, Youngjune Gwon, and Seungjai Min. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems , 33:21188-21198, 2020. 5

[KCY + 21] Yeong-Dae Kwon, Jinho Choo, Iljoo Yoon, Minah Park, Duwon Park, and Youngjune Gwon. Matrix encoding networks for neural combinatorial optimization. Advances in Neural Information Processing Systems , 34:5138-5149, 2021. 5

[KDZ + 17] Elias Khalil, Hanjun Dai, Yuyu Zhang, Bistra Dilkina, and Le Song. Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems , 30, 2017. 5

[KL20] Nikolaos Karalias and Andreas Loukas. Erdos goes neural: an unsupervised learning framework for combinatorial optimization on graphs. Advances in Neural Information Processing Systems , 33:6659-6672, 2020. 5

[KLMS19] Weiwei Kong, Christopher Liaw, Aranyak Mehta, and D Sivakumar. A new dog learns old tricks: Rl finds classic optimization algorithms. In International conference on learning representations , 2019. 5

[KP + 21] Minsu Kim, Jinkyoo Park, et al. Learning collaborative policies to solve np-hard routing problems. Advances in Neural Information Processing Systems , 34:10418-10430, 2021. 5

[KPP22] Minsu Kim, Junyoung Park, and Jinkyoo Park. Sym-nco: Leveraging symmetricity for neural combinatorial optimization. arXiv preprint arXiv:2205.13209 , 2022. 5

[KVHW18] Wouter Kool, Herke Van Hoof, and Max Welling. Attention, learn to solve routing problems! arXiv preprint arXiv:1803.08475 , 2018. 5

[LSS19] Jingcheng Liu, Alistair Sinclair, and Piyush Srivastava. The ising partition function: Zeros and deterministic approximation. Journal of Statistical Physics , 174(2):287-315, 2019. 27

[LSZ21] Troy Lee, Miklos Santha, and Shengyu Zhang. Quantum algorithms for graph problems with cut queries. In Proceedings of the 2021 ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 939-958. SIAM, 2021. 2

[LV16] Jasper CH Lee and Paul Valiant. Optimizing star-convex functions. In 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) , pages 603-614. IEEE, 2016. 7

[LZ09] Fei Liu and Guangzhou Zeng. Study of genetic algorithm with reinforcement learning to solve the tsp. Expert Systems with Applications , 36(3):6995-7001, 2009. 5

[MGH + 19] Qiang Ma, Suwen Ge, Danyang He, Darshan Thaker, and Iddo Drori. Combinatorial optimization by graph pointer networks and hierarchical reinforcement learning. arXiv preprint arXiv:1911.04936 , 2019. 5

[MKS + 13] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 , 2013. 5

[MLC + 21] Yining Ma, Jingwen Li, Zhiguang Cao, Wen Song, Le Zhang, Zhenghua Chen, and Jing Tang. Learning to iteratively solve routing problems with dual-aspect collaborative transformer. Advances in Neural Information Processing Systems , 34:11096-11107, 2021. 5

[MSIB21] Nina Mazyavkina, Sergey Sviridov, Sergei Ivanov, and Evgeny Burnaev. Reinforcement learning for combinatorial optimization: A survey. Computers & Operations Research , 134:105400, 2021. 5

[MYSSS21] Eran Malach, Gilad Yehudai, Shai Shalev-Schwartz, and Ohad Shamir. The connection between approximation, depth separation and learnability in neural networks. In Conference on Learning Theory , pages 3265-3295. PMLR, 2021. 5

[NJS + 20] Yatin Nandwani, Deepanshu Jindal, Parag Singla, et al. Neural learning of one-ofmany solutions for combinatorial problems in structured output spaces. arXiv preprint arXiv:2008.11990 , 2020. 5

[NOST18] Mohammadreza Nazari, Afshin Oroojlooy, Lawrence Snyder, and Martin Takác. Reinforcement learning for solving the vehicle routing problem. Advances in neural information processing systems , 31, 2018. 5

[PRW22] Orestis Plevrakis, Seyoon Ragavan, and S Matthew Weinberg. On the cut-query complexity of approximating max-cut. arXiv preprint arXiv:2211.04506 , 2022. 2

[PS98] Christos H Papadimitriou and Kenneth Steiglitz. Combinatorial optimization: algorithms and complexity . Courier Corporation, 1998. 1, 5

[RSW17] Aviad Rubinstein, Tselil Schramm, and S Matthew Weinberg. Computing exact minimum cuts without knowing the graph. arXiv preprint arXiv:1711.03165 , 2017. 2

[S + 03] Alexander Schrijver et al. Combinatorial optimization: polyhedra and efficiency , volume 24. Springer, 2003. 1, 5

[SBK22] Martin JA Schuetz, J Kyle Brubaker, and Helmut G Katzgraber. Combinatorial optimization with physics-inspired graph neural networks. Nature Machine Intelligence , 4(4):367-377, 2022. 5

[Sch05] Alexander Schrijver. On the history of combinatorial optimization (till 1960). Handbooks in operations research and management science , 12:1-68, 2005. 1, 5

[SE20] Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems , 33:12438-12448, 2020. 4

[SHM + 16] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature , 529(7587):484-489, 2016. 5

[Sin12] Alistair Sinclair. Algorithms for random generation and counting: a Markov chain approach . Springer Science & Business Media, 2012. 30, 33

[SK21] Yang Song and Diederik P Kingma. How to train your energy-based models. arXiv preprint arXiv:2101.03288 , 2021. 10

[SLB + 18] Daniel Selsam, Matthew Lamm, Benedikt Bünz, Percy Liang, Leonardo de Moura, and David L Dill. Learning a sat solver from single-bit supervision. arXiv preprint arXiv:1802.03685 , 2018. 5

[Smi99] Kate A Smith. Neural networks for combinatorial optimization: a review of more than a decade of research. Informs journal on Computing , 11(1):15-34, 1999. 1, 5

[SSDK + 20] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 , 2020. 4

[SSS + 17] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature , 550(7676):354-359, 2017. 5

[TRWG21] Jan Toenshoff, Martin Ritzert, Hinrikus Wolf, and Martin Grohe. Graph neural networks for maximum constraint satisfaction. Frontiers in artificial intelligence , 3:580607, 2021. 5

[VFJ15] Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. Advances in neural information processing systems , 28, 2015. 1, 5

[YBV19] Weichi Yao, Afonso S Bandeira, and Soledad Villar. Experimental performance of graph neural networks on random instances of max-cut. In Wavelets and Sparsity XVIII , volume 11138, pages 242-251. SPIE, 2019. 5

[YGS20] Gal Yehuda, Moshe Gabel, and Assaf Schuster. It's not what machines can learn, it's what we cannot teach. In International conference on machine learning , pages 10831-10841. PMLR, 2020. 5

[YP19] Emre Yolcu and Barnabás Póczos. Learning local search heuristics for boolean satisfiability. Advances in Neural Information Processing Systems , 32, 2019. 5

[YW20] Yunhao Yang and Andrew Whinston. A survey on reinforcement learning for combinatorial optimization. arXiv preprint arXiv:2008.12248 , 2020. 5

[ZCH + 20] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. AI Open , 1:57-81, 2020. 5

[ZMB + 17] Zhengyuan Zhou, Panayotis Mertikopoulos, Nicholas Bambos, Stephen Boyd, and Peter W Glynn. Stochastic mirror descent in variationally coherent optimization problems. Advances in Neural Information Processing Systems , 30, 2017. 7

A Preliminaries and Notation

This lemma is a useful tool for quasar convex functions.

Lemma 1 ([HMR16]) . Suppose that the functions 𝑓 1 , . . . , 𝑓 𝑛 are individually 𝛾 -quasar convex in 𝑋 with respect to a common global minimum 𝑥 . Then for non-negative weights 𝑎 1 , . . . , 𝑎 𝑛 , the linear combination 𝑓 = ∑︀ 𝑖 ∈ [ 𝑛 ] 𝑎 𝑖 𝑓 𝑖 is also 𝛾 -quasar convex with respect to 𝑥 in 𝑋 .

In the proofs, we use the following notation: for a matrix 𝑊 and vectors 𝑥, 𝑧 , we let

be a probability mass function over 𝑋 and we overload the notation as

B The Proof of Remark 1

Proof. Let 𝑥 1 , . . . , 𝑥 𝑛 be the variables of the Max-Cut problem of interest and 𝑆 = {-1 , 1 } be the solution space. Consider 𝒫 to be the collection of product distributions over 𝑆 , i.e., for any 𝑝 ∈ 𝒫 , it holds that, for any 𝑠 ∈ 𝑆 , Pr 𝑥 ∼ 𝑝 [ 𝑥 = 𝑠 ] = ∏︀ 𝑖 ∈ [ 𝑛 ] 𝑝 1+ 𝑠 𝑖 2 𝑖 (1 -𝑝 𝑖 ) 1 -𝑠 𝑖 2 . Let us consider the cube [ 𝜖, 1 -𝜖 ] 𝑛 . This family is complete since the 𝑂 ( 𝜖 ) -sub-optimal solution of 𝐼 belongs to 𝒫 and is compressed since the description size is poly ( 𝑛, log(1 /𝜖 )) . We show that in this setting there exist bad stationary points. Let 𝐿 𝐺 be the Laplacian matrix of the input graph. For some product distribution 𝑝 ∈ 𝒫 , it holds that

𝑛

where 𝐿 𝐺 is zero in the diagonal and equal to the Laplacian otherwise. Let us consider a vertex of the cube 𝑝 ∈ [ 𝜖, 1 -𝜖 ] 𝑛 which is highly and strictly sub-optimal, i.e., any single change of a node would strictly improve the number of edges in the cut and the score attained in 𝑝 is very large compared to min 𝑥 ∈ 𝑆 -𝑥 ⊤ 𝐿 𝐺 𝑥 . For any 𝑖 ∈ [ 𝑛 ] , we show that

This means that if 𝑝 𝑖 is large (i.e., 1 -𝜖 ), then the 𝑖 -th coordinate of the gradient of ℒ ( 𝑝 ) should be negative since this would imply that the negative gradient would preserve 𝑝 𝑖 to the right boundary. Similarly for the case where 𝑝 𝑖 is small. This means that this point is a stationary point and is highly sub-optimal by assumption.

Let 𝑃 (resp. 𝑁 ) be the set of indices in [ 𝑛 ] where 𝑝 takes the value 1 -𝜖 (resp. 𝜖 ). For any 𝑖 ∈ [ 𝑛 ] , let 𝒩 ( 𝑖 ) be its neighborhood in 𝐺 . Let us consider 𝑖 ∈ 𝑃 . We have that (2 𝑝 -1) · 𝑒 𝑖 > 0 and so it suffices to show that

which corresponds to showing that

and so we would like to have

Note that this is true for any 𝑖 ∈ [ 𝑛 ] since the current solution is a strict local optimum. The same holds if 𝑖 ∈ 𝑁 .

C Completeness

Proposition 2 (Completeness) . Consider 𝜖 > 0 and a prior ℛ over ℐ . Assume that Assumption 1 holds. There exist 𝛽 ⋆ , 𝜌 ⋆ ∈ (0 , 1) and 𝒲 such that the family of solution generators 𝒫 of Equation (5) is complete.

Proof. Assume that 𝒪 ( 𝑠, 𝐼 ) = 𝜓 ℐ ( 𝐼 ) ⊤ 𝑀𝜓 𝑆 ( 𝑠 ) and let 𝑧 = 𝜓 ℐ ( 𝐼 ) and 𝑥 = 𝜓 𝑆 ( 𝑠 ) . Moreover, let 𝛼, 𝐶, 𝐷 𝑆 , 𝐷 ℐ be the parameters promised by Assumption 1. Let us consider the family 𝒫 = { 𝑝 ( 𝑊 ) : 𝑊 ∈ 𝒲} with

where the mixing weight 𝛽 ⋆ ∈ (0 , 1) and the inverse temperate 𝜌 ⋆ are to be decided. Recall that

Let us pick the parameter matrix 𝑊 = -𝑀/𝜆 . Let us now fix a 𝑧 ∈ 𝜓 ℐ ( ℐ ) . For the given matrix 𝑀 , we can consider the finite set of values 𝑉 obtained by the quadratic forms { 𝑧 ⊤ 𝑀𝑥 } 𝑥 ∈ 𝜓 𝑆 ( 𝑆 ) . We further cluster these values so that they have distance at least 𝜖 between each other. We consider the level sets 𝐶 𝑖 where 𝐶 1 is the subset of 𝑆 with minimum value 𝑣 1 (= 𝑣 1 ( 𝑧 )) ∈ 𝑉 , 𝐶 2 is the subset with the second smallest 𝑣 2 (= 𝑣 2 ( 𝑧 )) ∈ 𝑉 , etc. For fixed 𝑧 ∈ 𝜓 ℐ ( ℐ ) , we have that

where 𝜑 comes from (1). We note that

We claim that, by letting 𝜆 → 0 , the above measure concentrates uniformly on 𝐶 1 . The worst case scenario is when | 𝐶 2 | = | 𝑆 | - | 𝐶 1 | and 𝑣 2 = 𝑣 1 + 𝜖 . Then we have that

when 1 /𝜆 > log( | 𝜓 𝑆 ( 𝑆 ) | /𝛿 ) /𝜖 , since in the worst case | 𝐶 2 | / | 𝐶 1 | = Ω( | 𝜓 𝑆 ( 𝑆 ) | ) . Using this choice of 𝜆 and taking expectation over 𝑧 , we get that

First, we remark that by taking 𝜌 ⋆ = poly ( 𝛼, 1 /𝐶, 1 /𝐷 𝑆 , 1 /𝐷 ℐ ) , the last term in the right-hand side of the above expression can be replaced by the expected score of an almost-uniform solution (see Lemma 3 and Proposition 8), which is at most poly ( 𝐷 𝑆 , 𝐷 ℐ , 𝐶 )2 -| 𝜓 𝑆 ( 𝑆 ) | (and which is essentially negligible). Finally, one can pick 𝛽 ⋆ , 𝛿 = poly ( 𝜖, 1 /𝐶, 1 /𝐷 𝑆 , 1 /𝐷 ℐ ) so that

This implies that 𝒫 is complete by letting 𝑊 = -𝑀/𝜆 ∈ 𝒲 . This means that one can take 𝒲 be a ball centered at 0 with radius (of 𝜖 -sub-optimality) to be of order at least 𝐵 = ‖ 𝑀 ‖ F /𝜆.

D Compression

Proposition 3 (Compression) . Consider 𝜖 > 0 and a prior ℛ over ℐ . Assume that Assumption 1 holds. There exist 𝛽 ⋆ , 𝜌 ⋆ ∈ (0 , 1) and 𝒲 such that the family of solution generators 𝒫 of Equation (5) is compressed.

Proof. We have that the bit complexity to represent the mixing weight 𝛽 ⋆ is polylog ( 𝐷 𝑆 , 𝐷 ℐ , 𝐶, 1 /𝜖 ) and the description size of 𝒲 is polynomial in [ ℐ ] and in log(1 /𝜖 ) . This follows from Assumption 1 since the feature dimensions 𝑛 𝑋 and 𝑛 𝑍 are poly ([ ℐ ]) and 𝒲 is a ball centered at 0 with radius 𝑂 ( 𝐵 ) , where 𝐵 = ‖ 𝑀 ‖ F /𝜆 ≤ 𝐶/𝜆 , which are also poly ([ ℐ ] /𝜖 ) .

E Efficiently Optimizable

Proposition 4 (Efficiently Optimizable) . Consider 𝜖 > 0 and a prior ℛ over ℐ . Assume that Assumption 1 holds. There exist 𝛽 ⋆ , 𝜌 ⋆ ∈ (0 , 1) and 𝒲 such that family of solution generators 𝒫 of Equation (5) is efficiently optimizable using Projected SGD, where the projection set is 𝒲 .

The proof of this proposition is essentially decomposed into two parts: first, we show that the entropy-reularized loss of Equation (2) is quasar convex and then apply the projected SGD algorithm to ℒ 𝜆 .

Recall that 𝐻 ( 𝑊 ) = E 𝑧 ∼ℛ E 𝑥 ∼ 𝜑 ( · ; 𝑧 ; 𝑊 ) [log 𝜑 ( 𝑥 ; 𝑧 ; 𝑊 )] . Let 𝑅 be a weighted sum (to be in accordance with the mixture structure of 𝒫 ) of negative entropy regularizers

where 𝛽 ⋆ , 𝜌 ⋆ are the fixed parameters of 𝒫 (recall Equation (5)). We define the regularized loss

E.1 Quasar Convexity of the Regularized Loss

In this section, we show that ℒ 𝜆 of Equation (2) is quasar convex. We restate Proposition 1.

Proposition 5 (Quasar Convexity) . Consider 𝜖 > 0 and a prior ℛ over ℐ . Assume that Assumption 1 holds. The function ℒ 𝜆 of Equation (2) with domain 𝒲 is poly ( 𝐶, 𝐷 𝑆 , 𝐷 ℐ , 1 /𝜖, 1 /𝛼 ) -quasar convex with respect to -𝑀/𝜆 on the domain 𝒲 .

Proof. We can write the loss ℒ 𝜆 as

where the mappings ℒ 𝑧 and 𝑅 𝑧 are instance-specific (i.e., we have fixed 𝑧 ). We can make use of Lemma 1, which states that linear combinations of quasar convex (with the same minimizer) remain quasar convex. Hence, since the functions ℒ 𝜆,𝑧 have the same minimizer -𝑀/𝜆 , it suffices to show quasar convexity for a particular fixed instance mapping, i.e., it sufffices to show that the function

is quasar convex. Recall that 𝑊 is a matrix of dimension 𝑛 𝑍 × 𝑛 𝑋 . To deal with the function ℒ 𝜆,𝑧 , we consider the simpler function that maps vectors instead of matrices to real numbers. For some vector 𝑐 , let ℒ vec 𝜆 : R 𝑛 𝑋 → R be

where for any vector 𝑤 ∈ R 𝑛 𝑋 , we define the probability distribution 𝜑 ( · ; 𝑤 ) over the solution space 𝑋 = 𝜓 𝑆 ( 𝑆 ) with probability mass function

We then define and 𝑅 vec ( 𝑤 ) = (1 -𝛽 ⋆ ) 𝐻 ( 𝑤 ) + 𝛽 ⋆ 𝜌 ⋆ 𝐻 ( 𝜌 ⋆ 𝑤 ) (this is a essentially a weighted sum of regularizers, needed to simplify the proof) with 𝐻 ( 𝑤 ) = E 𝑥 ∼ 𝜑 ( · ; 𝑤 ) log 𝜑 ( 𝑥, 𝑤 ) . These quantities are essentially the fixed-instance analogues of Equations (5) and (2). The crucial observation is that by taking 𝑐 = 𝑧 ⊤ 𝑀 and applying the chain rule we have that

where

This means that the gradient of the fixed-instance objective ℒ 𝜆,𝑧 is a matrix of dimension 𝑛 𝑍 × 𝑛 𝑋 that is equal to the outer product of the instance featurization 𝑧 and the gradient of the simpler function ℒ vec 𝑤 evaluated at 𝑧 ⊤ 𝑊 . Let us now return on showing that ℒ 𝜆,𝑧 is quasar convex. To this end, we observe that

This means that, since 𝑧 is fixed, it suffices to show that the function ℒ vec 𝜆 is quasar convex. We provide the next key proposition that deals with issue. This result is one the main technical aspects of this work and its proof can be found in Appendix E.2.

In the following, intuitively 𝒳 is the post-featurization instance space and 𝒵 is the parameter space.

Proposition 6. Consider 𝜖, 𝜆 > 0 . Let ‖ 𝑐 ‖ 2 ≤ 𝐶 1 . Let 𝒵 be an open ball centered at 0 with diameter 2 𝐶 1 /𝜆 . Let 𝒳 be a space of diameter 𝐷 and let Var 𝑥 ∼ 𝑈 ( 𝒳 ) [ 𝑣 · 𝑥 ] ≥ 𝛼 ‖ 𝑣 ‖ 2 2 for any 𝑣 ∈ 𝒵 . The function ℒ vec 𝜆 ( 𝑤 ) = E 𝑥 ∼ 𝑝 ( · ; 𝑤 ) [ 𝑐 · 𝑥 ] + 𝜆𝑅 vec ( 𝑤 ) is poly (1 /𝐶 1 , 1 /𝐷,𝜖, 𝛼 ) -quasar convex with respect to -𝑐/𝜆 on 𝒵 .

We can apply the above result with 𝑐 = 𝑧 ⊤ 𝑀,𝑤 = 𝑧 ⊤ 𝑊,𝐷 = 𝐷 𝑆 and 𝐶 1 = 𝐷 ℐ 𝐶 . These give that the quasar convexity parameter 𝛾 is of order 𝛾 = poly ( 𝜖, 𝛼, 1 /𝐶, 1 /𝐷 𝑆 , 1 /𝐷 ℐ ) . Since we have that ℒ vec 𝜆 ( 𝑧 ⊤ 𝑊 ) = ℒ 𝜆,𝑧 ( 𝑊 ) , we get that

This implies that ℒ 𝜆,𝑧 is 𝛾 -quasar convex with respect to the minimizer -𝑀/𝜆 and completes the proof using Lemma 1.

E.2 The Proof of Proposition 6

Let us consider ℒ vec 𝜆 to be a real-valued differentiable function defined on 𝒵 . Let 𝑤, -𝑐/𝜆 ∈ 𝒵 and let 𝐿 be the line segment between them with 𝐿 ∈ 𝒵 . The mean value theorem implies that there exists 𝑤 ′ ∈ 𝐿 such that

Now we have that ℒ vec 𝜆 has bounded gradient (see Lemma 2) and so we get that

since 𝑤 ′ ∈ 𝐿 . This implies that

where 1 /𝛾 = poly ( 𝐶 1 ,𝐷 ) 𝜖 3 𝛼 2 . The last inequality is an application of the correlation lower bound (see Lemma 3).

In the above proof, we used two key lemmas: a bound for the norm of the gradient and a lower bound for the correlation. In the upcoming subsections, we prove these two results.

E.2.1 Bounded Gradient Lemma and Proof

Lemma 2 (Bounded Gradient Norm of ℒ vec 𝜆 ) . Consider 𝜖, 𝜆 > 0 . Let 𝒵 be the domain of ℒ vec 𝜆 of (3) . Let 𝒳 be a space of diameter 𝐷 . For any 𝑤 ∈ 𝒵 , it holds that

Proof. We have that

where

and

Note that since 𝑥 ∈ 𝒳 , it holds that ‖ 𝑥 ‖ 2 ≤ 𝐷 . Hence

Moreover, we have that

This means that

E.2.2 Correlation Lower Bound Lemma and Proof

The following lemma is the second ingredient in order to show Proposition 6.

Lemma 3 (Correlation Lower Bound for ℒ vec 𝜆 ) . Let 𝜆 > 0 . Let ‖ 𝑐 ‖ 2 ≤ 𝐶 1 . Let 𝒵 be an open ball centered at 0 with diameter 𝐵 = 2 𝐶 1 /𝜆 . Let 𝒳 be a space of diameter 𝐷 . Assume that 𝑤 ∈ 𝒵 and Var 𝑥 ∼ 𝑈 ( 𝒳 ) [( 𝑐 + 𝜆𝑤 ) · 𝑥 ] ≥ 𝛼 ‖ 𝑐 + 𝜆𝑤 ‖ 2 2 for some 𝛼 > 0 . Then, for any 𝛽 ⋆ ∈ (0 , 1) , there exists 𝜌 ⋆ > 0 such that it holds that

where ℒ vec 𝜆 is the regularized loss of Proposition 6, 𝜌 ⋆ is the scale in the second component of the mixture of (5) and 𝛽 ⋆ ∈ (0 , 1) is the mixture weight.

First, in Lemma 4 and Lemma 5, we give a formula for the desired correlation ∇ 𝑤 ℒ vec 𝜆 ( 𝑤 ) · ( 𝑐 + 𝜆𝑤 ) and, then we can provide a proof for Lemma 3 by lower bounding this formula.

Lemma4 (Correlation with Regularization) . Consider the function 𝑔 ( 𝑤 ) = E 𝑥 ∼ 𝜑 ( · ; 𝑤 ) [ 𝑐 · 𝑥 ]+ 𝜆𝐻 ( 𝑤 ) , where 𝐻 is the negative entropy regularizer. Then it holds that

Proof. Let us consider the following objective function:

where 𝐻 is the negative entropy regularizer, i.e.,

The gradient of 𝑔 with respect to 𝑤 ∈ 𝒲 is equal to

It holds that and

So, we get that

Note that

Lemma 5 (Gradient with Regularization and Mixing) . For any 𝜖 > 0 , for the family of solution generators 𝒫 = { 𝑝 ( · ; 𝑤 ) = (1 -𝛽 ⋆ ) 𝜑 ( · ; 𝑤 ) + 𝛽 ⋆ 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) : 𝑤 ∈ 𝒲} and the objective ℒ vec 𝜆 of Equation (3) , it holds that

Proof. Let us first consider the scaled parameter 𝜌𝑤 ∈ 𝒲 for some 𝜌 > 0 . Then it holds that

Moreover, the negative entropy regularizer at 𝜌𝑤 is

It holds that

We consider the objective function ℒ vec 𝜆 to be defined as follows: first, we take

i.e., 𝑝 ( · ; 𝑤 ) is the mixture of the probability measures 𝜑 ( · ; 𝑤 ) and 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) with weights 1 -𝛽 ⋆ and 𝛽 ⋆ respectively for some scale 𝜌 ⋆ > 0 . Moreover, we take 𝑅 vec ( 𝑤 ) = (1 -𝛽 ⋆ ) 𝐻 ( 𝑤 ) + 𝛽 ⋆ 𝜌 ⋆ 𝐻 ( 𝜌 ⋆ 𝑤 ) . Then we define our regularized loss ℒ vec to be

𝜆

Using Lemma 4 and the above calculations, we have that

The above calculations yield

The above correlation being positive intuitively means that performing gradient descent to ℒ vec 𝜆 gives that the parameter 𝑤 converges to -𝑐/𝜆 , the point that achieves completeness for that objective.

However, to obtain fast convergence, we need to show that the above correlation is non-trivial. This means that our goal in order to prove Lemma 3 is to provide a lower bound for the above quantity, i.e., it suffices to give a non-trivial lower bound for the variance of the random variable ( 𝑐 + 𝜆𝑤 ) · 𝑥 with respect to the probability measure 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) . It is important to note that in the above statement we did not fix the value of 𝜌 ⋆ . We can now make use of Proposition 8. Intuitively, by taking the scale parameter appearing in the mixture 𝜌 ⋆ to be sufficiently small, we can manage to provide a lower bound for the variance of ( 𝑐 + 𝜆𝑤 ) · 𝑥 with respect to the almost uniform measure, i.e, the second summand of the above right-hand side expression has significant contribution. We remark that 𝜌 corresponds to the inverse temperature parameter. Hence, our previous analysis essentially implies that policy gradient on combinatorial optimization potentially works if the variance Var 𝑥 ∼ 𝜑 ( · ; 𝜌𝑤 ) [( 𝑐 + 𝜆𝑤 ) · 𝑥 ] is non-vanishing at high temperatures 1 /𝜌 .

The proof of Lemma 3. Recall that

where the scale parameter 𝜌 > 0 is to be decided. This means that

Our goal is now to apply Proposition 8 in order to lower bound the above variance. Applying Proposition 8 for 𝜇 ← 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) , 𝑐 ← 𝑐 + 𝜆𝑤 ∈ 𝒵 and, so for some absolute constant 𝐶 0 , we can pick

Thus, we have that

This implies the desired result since

E.3 Convergence for Quasar Convex Functions

The fact that ℒ 𝜆 is quasar convex with respect to -𝑀/𝜆 implies that projected SGD converges to that point in a small number if steps and hence the family 𝒫 is efficiently optimizable. The analysis is standard (see e.g., [HMR16]). For completeness a proof can be found in Appendix E.3.

Proposition 7 (Convergence) . Consider 𝜖 > 0 and a prior ℛ over ℐ . Assume that Assumption 1 holds with parameters 𝐶, 𝐷 𝑆 , 𝐷 ℐ , 𝛼 . Let 𝑊 1 , . . . , 𝑊 𝑇 be the updates of the SGD algorithm with projection set 𝒲 performed on ℒ 𝜆 of Equation (2) with appropriate step size and parameter 𝜆 . Then, for the non-regularized objective ℒ , it holds that

when 𝑇 ≥ poly (1 /𝜖, 1 /𝛼, 𝐶, 𝐷 𝑆 , 𝐷 ℐ , ‖ 𝑊 0 + 𝑀/𝜆 ‖ F ) .

Our next goal is to use Proposition 1 and show that standard projected SGD on the objective ℒ 𝜆 converges in a polynomial number of steps. The intuition behind this result is that since the correlation between ∇ℒ 𝜆 ( 𝑊 ) and the direction 𝑀 + 𝜆𝑊 is positive and non-trivial, the gradient field drives the optimization method towards the point -𝑀/𝜆 .

Proof. Consider the sequence of matrices 𝑊 1 , . . . , 𝑊 𝑡 , . . . , 𝑊 𝑇 generated by applying PSGD on ℒ 𝜆 with step size 𝜂 (to be decided) and initial parameter vector 𝑊 0 ∈ 𝒲 . We have that ℒ 𝜆 is 𝛾 -quasar convex and is also 𝑂 (Γ) -weakly smooth 2 since we now show that it is Γ -smooth.

Lemma 6. ℒ 𝜆 is poly ( 𝐷 𝑆 , 𝐷 ℐ , 𝐶 ) -smooth.

Proof. We have that

It suffices to show that ℒ vec 𝜆 is smooth. Recall that

This means that where

Standard computation of these values yields that, since 𝐷 𝑆 and 𝐷 ℐ are bounds to 𝑥 and 𝑧 respectively, we have that ℒ 𝜆 is smooth with parameter poly ( 𝐷 𝑆 , 𝐷 ℐ , 𝐶 ) .

Let 𝑉 be the variance of the unbiased estimator used for ∇ 𝑊 ℒ 𝜆 ( 𝑊 ) . We can apply the next result of [HMR16].

Lemma 7 ([HMR16]) . Suppose the objective function 𝑓 is 𝛾 -weakly quasi convex and Γ -weakly smooth, and let 𝑟 ( · ) be an unbiased estimator for ∇ 𝑓 ( 𝜃 ) with variance 𝑉 . Moreover, suppose the global minimum 𝜃 belongs to 𝒲 , and the initial point 𝜃 0 satisfies ‖ 𝜃 0 -𝜃 ‖ 2 ≤ 𝑅 . Then projected stochastic gradient descent with a proper learning rate returns 𝜃 𝑇 in 𝑇 iterations with expected error

Weapply the above result to ℒ 𝜆 in order to find matrices 𝑊 1 , ..., 𝑊 𝑇 that achieve good loss on average compared to -𝑀/𝜆 . Moreover, using a batch SGD update, we can take 𝑉 to be also polynomial in the crucial parameters of the problem. We note that one can adapt the above convergence proof and show that the actual loss ℒ (and not the loss ℒ 𝜆 ) are close after sufficiently many iterations (as indicated by the above lemma). We know that the Frobenius norm of the gradient of ℒ ( 𝑊 ) is at most of order 𝑂 ( 𝐷 2 ℐ 𝐶𝐷 2 𝑆 ) . We can apply the mean value theorem in high dimensions (by taking 𝒲 to be an open ball of radius 𝑂 ( 𝐵 ) ) and this yields that the difference between the values of ℒ ( 𝑊 𝑇 ) and ℒ ( -𝑀/𝜆 ) is at most 𝐷 2 ℐ 𝐶𝐷 2 𝑆 ‖ 𝑊 𝑡 + 𝑀/𝜆 ‖ 2 F . However, the right-hand side is upper bounded by the correlation between ∇ℒ 𝜆 ( 𝑊 𝑡 ) and 𝑊 𝑡 + 𝑀/𝜆 . Hence, we can still use this correlation as a potential in order to minimize ℒ . This implies that the desired convergence guarantee holds as long as 𝑇 ≥ poly (1 /𝜖, 1 /𝛼, 𝐶, 𝐷 𝑆 , 𝐷 ℐ , ‖ 𝑊 0 + 𝑀/𝜆 ‖ F ) .

F Deferred Proofs: Variance under Almost Uniform Distributions

This section is a technical section that states some properties of exponential families. We use some standard notation, such as 𝑤 and 𝑥 , for the statements and the proofs but we underline that these symbols do not correspond to the notation in the main body of the paper.

We consider the parameter space Θ and for any parameter 𝑤 ∈ Θ , we define the probability distribution 𝜑 ( · ; 𝑤 ) over a space 𝒳 with density

In this section, our goal is to relate the variance of 𝑐 · 𝑥 under the measure 𝜑 ( · ; 0) (uniform case) and 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) for some 𝑤 ∈ 𝒲 and some sufficiently small 𝜌 ⋆ (almost uniform case). The main result of this section follows.

Proposition 8 (Variance Lower Bound Under Almost Uniform Distributions) . Assume that the variance of 𝑐 · 𝑥 under the uniform distribution over 𝒳 , whose diameter is 𝐷 , is lower bounded by 𝛼 ‖ 𝑐 ‖ 2 2 . Moreover assume that 𝑤 ∈ Θ with ‖ 𝑤 ‖ 2 ≤ 𝐵 . Then, setting 𝜌 ⋆ = 𝑂 ( 𝛼/ ( 𝐵𝐷 3 )) , it holds that Var 𝑥 ∼ 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) [ 𝑐 · 𝑥 ] = Ω( 𝛼 ‖ 𝑐 ‖ 2 2 ) .

We first provide a general abstract lemma that relates the variance of the uniform distribution 𝑈 over 𝒳 to the variance of an almost uniform probability measure 𝜇 . For simplicity, we denote the uniform distribution over 𝒳 with 𝑈 = 𝑈 ( 𝒳 ) .

Lemma 8. Let 𝑤 ∈ Θ and 𝑥 ∈ 𝒳 with ‖ 𝑥 ‖ 2 ≤ 𝐷 . Consider the uniform probability measure 𝑈 over 𝒳 and let 𝜇 over 𝒳 be such that there exist 𝜖 1 , 𝜖 2 > 0 with:

Then it holds that Var 𝑥 ∼ 𝜇 [ 𝑤 · 𝑥 ] ≥ Var 𝑥 ∼ 𝑈 [ 𝑤 · 𝑥 ] -3 max { 𝜖 2 1 , 𝜖 1 𝐷,𝜖 2 }‖ 𝑤 ‖ 2 2 .

Proof. We have that

We first deal with upper-bounding the square of the first moment. Note that

Let us take 𝜖 > 0 (with 𝜖 < 𝜖 1 ) for simplicity to be such that ( E 𝑥 ∼ 𝜇 [ 𝑤 · 𝑥 ]) 2 = ( E 𝑥 ∼ 𝑈 [ 𝑤 · 𝑥 ] + 𝜖 ‖ 𝑤 ‖ 2 ) 2 . This means that

Next we lower-bound the second moment. It holds that

for some 𝜖 2 > 0 . This means that

Hence,

Our next goal is to relate 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) with the uniform measure 𝜑 ( · ; 0) . According to the above general lemma, we have to relate the first and second moments of 𝜑 ( · ; 𝜌 ⋆ 𝑤 ) with the ones of the uniform distribution 𝑈 = 𝜑 ( · ; 0) .

The Proof of Proposition 8. Our goal is to apply Lemma 8. First, let us set

for any unit vector 𝑣 ∈ Θ . Then it holds that

Using the mean value theorem in [0 , 𝜌 ⋆ ] for any unit vector 𝑣 , we have that there exists a 𝜉 = 𝜉 𝑣 ∈ (0 , 𝜌 ⋆ ) such that

It suffices to upper bound 𝑓 ′ 𝑣 ( 𝜉 ) for any unit vector 𝑣 and 𝜉 ∈ (0 , 𝜌 ⋆ ) . Let us compute 𝑓 ′ 𝑣 . We have that

Since 𝑥 ∈ 𝒳 , we have that

This gives that

We then continue with controlling the second moment: it suffices to find 𝜖 2 such that for any 𝑣 ∈ Θ , it holds

Let us set 𝑔 𝑣 ( 𝜌 ) = E 𝑥 ∼ 𝜑 ( · ; 𝜌𝑤 ) [( 𝑣 · 𝑥 ) 2 ] for any vector 𝑣 ∈ Θ . We have that

where 𝜉 ∈ (0 , 𝜌 ⋆ ) . It holds that

This gives that for any 𝑣 ∈ Θ , it holds

Note that the above holds for 𝑣 = 𝑐 too. Lemma 8 gives us that

where 𝜖 1 = 2 𝜌 ⋆ 𝐵𝐷 2 and 𝜖 2 = 2 𝜌 ⋆ 𝐵𝐷 3 . This implies that by picking

for some universal constant 𝐶 0 , we get that

In this section, we considered 𝜌 ⋆ as indicated by the above Proposition 8.

G Applications to Combinatorial Problems

In this section we provide a series of combinatorial applications of our theoretical framework (Theorem 1). In particular, for each one of the following combinatorial problems (that provably satisfy Assumption 1), it suffices to specify the feature mappings 𝜓 𝑆 , 𝜓 ℐ and compute the parameters 𝐶, 𝐷 𝑆 , 𝐷 ℐ , 𝛼 .

G.1 Maximum Cut, Maximum Flow and Max𝑘 -CSPs

We first provide a general lemma for the variance of 'linear tensors' under the uniform measure.

Lemma 9 (Variance Lower Bound Under Uniform) . Let 𝑛, 𝑘 ∈ N . For any 𝑤 ∈ R ( 𝑛 𝑘 ) , it holds that

Proof. For any 𝑤 ∈ R ( 𝑛 𝑘 ) , it holds that

Note that 𝑤 ∈ R ( 𝑛 𝑘 ) can be written as 𝑤 = ( 𝑤 ∅ , 𝑤 -∅ ) where 𝑤 ∅ corresponds to the constant term of the Fourier expansion and 𝑤 -∅ = ( 𝑤 𝑆 ) ∅̸ = 𝑆 ⊆ [ 𝑛 ]: | 𝑆 |≤ 𝑘 is the vector of the remaining coordinates. The Fourier expansion implies that

which yields the desired equality for the variance.

G.1.1 Maximum Cut

Let us consider a graph with 𝑛 nodes and weighted adjacency matrix 𝐴 with non-negative weights. Maximum cut is naturally associated with the Ising model and, intuitively, our approach does not yield an efficient algorithm for solving Max-Cut since we cannot efficiently sample from the Ising model in general. To provide some further intuition, consider a single-parameter Ising model for 𝐺 = ( 𝑉, 𝐸 ) with Hamiltonian 𝐻 𝐺 ( 𝑥 ) = ∑︀ ( 𝑖,𝑗 ) ∈ 𝐸 1+ 𝑥 𝑖 𝑥 𝑗 2 . Then the partition function is equal to 𝑍 𝐺 ( 𝛽 ) = ∑︀ 𝑥 ∈{-1 , 1 } 𝑉 exp( 𝛽𝐻 𝐺 ( 𝑥 )) . Note that when 𝛽 > 0 , the Gibbs measure favours configurations with alligned spins (ferromagnetic case) and when 𝛽 < 0 , the measure favours configurations with opposite spins (anti-ferromagnetic case). The antiferromagnetic Ising model appears to be more challenging. According to physicists the main reason is that its Boltzmann distribution is prone to a complicated type of long-range correlation known as 'replica symmetry breaking' [COLMS22]. From the TCS viewpoint, observe that as 𝛽 goes to -∞ , the mass of the Gibbs distribution shifts to spin configurations with more edges joining vertices with opposite spins and concentrates on the maximum cuts of the graph. Hence, being able to efficiently approximate the log-partition function for general Ising models, would lead to solving the Max-Cut problem.

Theorem 2 (Max-Cut has a Compressed and Efficiently Optimizable Solution Generator) . Consider a prior over Max-Cut instances with 𝑛 nodes. For any 𝜖 > 0 , there exists a solution generator 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} such that 𝒫 is complete, compressed with description poly ( 𝑛 ) polylog (1 /𝜖 ) and ℒ + 𝜆𝑅 : 𝒲 ↦→ R is efficiently optimizable via projected stochastic gradient descent in poly ( 𝑛, 1 /𝜖 ) steps for some 𝜆 > 0 .

Proof of Theorem 2. It suffices to show that Max-Cut satisfies Assumption 1. Consider an input graph 𝐺 with 𝑛 nodes and Laplacian matrix 𝐿 𝐺 . Then

We show that there exist feature mappings so that the cost of every solution 𝑠 under any instance/graph 𝐺 is a bilinear function of the feature vectors (cf. Item 2 of Assumption 1). We consider the correlationbased feature mapping 𝜓 𝑆 ( 𝑠 ) = ( 𝑠𝑠 ⊤ ) ♭ ∈ R 𝑛 2 , where by ( · ) ♭ we denote the vectorization/flattening operation and the negative Laplacian for the instance (graph), 𝜓 ℐ ( 𝐺 ) = ( -𝐿 𝐺 ) ♭ ∈ R 𝑛 2 . Then simply setting the matrix 𝑀 to be the identity 𝐼 ∈ R 𝑛 2 × 𝑛 2 the cost of any solution 𝑠 can be expressed as the bilinear function 𝜓 ℐ ( 𝐺 ) ⊤ 𝑀𝜓 𝑆 ( 𝑠 ) = ( -𝐿 ♭ 𝐺 ) ⊤ ( 𝑠𝑠 𝑇 ) ♭ = -𝑠 ⊤ 𝐿 𝐺 𝑠 . We observe that (for unweighted graphs) with 𝑛 nodes the bit-complexity of the family of all instances ℐ is roughly 𝑂 ( 𝑛 2 ) , and therefore the dimensions of the 𝜓 𝑆 , 𝜓 ℐ feature mappings are clearly polynomial in the bit-complexity of ℐ . Moreover, considering unweighted graphs, it holds ‖ 𝜓 ℐ ( 𝐺 ) ‖ 2 , ‖ 𝜓 𝑆 ( 𝑠 ) ‖ 2 , ‖ 𝑀 ‖ F ≤ poly ( 𝑛 ) . Therefore, the constants 𝐷 𝑆 , 𝐷 ℐ , 𝐶 are polynomial in the bit-complexity of the instance family.

It remains to show that our solution feature mapping satisfy the variance preservation assumption. For any 𝑣 , we have that Var 𝑠 ∼ 𝑈 ( 𝑆 ) [ 𝑣 · 𝜓 𝑆 ( 𝑠 )] = Var 𝑠 ∼ 𝑈 ( {-1 , 1 } 𝑛 ) [ 𝑣 · ( 𝑠𝑠 ⊤ ) ♭ ] = Ω( ‖ 𝑣 ‖ 2 2 ) , using Lemma 9 with 𝑘 = 2 , since 𝑐 ∅ = 0 with loss of generality.

G.1.2 Minimum Cut/Maximum Flow

Let us again consider a graph with 𝑛 nodes and Laplacian matrix 𝐿 𝐺 . It is known that the minimum cut problem is solvable in polynomial time when all the weights are positive. From the discussion of the maximum cut case, we can intuitively relate minimum cut with positive weights to the ferromagnetic Ising setting [dPS97]. We remark that we can consider the ferromagnetic parameter space 𝒲 fer = R ( 𝑛 2 ) ≥ 0 and get the variance lower bound from Lemma 9. We constraint projected SGD in 𝒲 fer. This means that during any step of SGD our algorithm has to sample from a mixture of ferromagnetic models with known mixture weights. The state of the art approximate sampling algorithm from ferromagnetic Ising models achieves the following performance, improving on prior work [JS93, LSS19, CLV22, CGG + 19].

Proposition 9 (Theorem 1.1 of [CZ22]) . Let 𝛿 𝛽 , 𝛿 𝜆 ∈ (0 , 1) be constants and 𝜇 be the Gibbs distribution of the ferromagnetic Ising model specified by graph 𝐺 = ( 𝑉, 𝐸 ) , | 𝑉 | = 𝑛, | 𝐸 | = 𝑚 , parameters 𝛽 ∈ [1 + 𝛿 𝛽 , + ∞ ) 𝑚 and external field 𝜆 ∈ [0 , 1 -𝛿 𝜆 ] 𝑛 . There exists an algorithm that

samples 𝑋 satisfying TV( 𝑋,𝜇 ) ≤ 𝜖 for any given parameter 𝜖 ∈ (0 , 1) within running time

.

This algorithm can handle general instances and it only takes a near-linear running time when parameters are bounded away from the all-ones vector. Our goal is to sample from a mixture of two such ferromagnetic Ising models which can be done efficiently. For simplicity, we next restrict ourselves to the unweighted case.

Theorem 3 (Min-Cut has a Compressed, Efficiently Optimizable and Samplable Solution Generator) . Consider a prior over Min-Cut instances with 𝑛 nodes. For any 𝜖 > 0 , there exists a solution generator 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} such that 𝒫 is complete, compressed with description poly ( 𝑛 ) polylog (1 /𝜖 ) , ℒ + 𝜆𝑅 : 𝒲 ↦→ R is efficiently optimizable via projected stochastic gradient descent in poly ( 𝑛, 1 /𝜖 ) steps for some 𝜆 > 0 and efficiently samplable in poly ( 𝑛, 1 /𝜖 ) steps.

Proof. We have that

The analysis (i.e., the selection of the feature mappings) is similar to the one of Theorem 2 with the sole difference that the parameter space is constrained to be 𝒲 fer and 𝜓 ℐ ( 𝐺 ) = ( 𝐿 𝐺 ) ♭ . We note that Proposition 9 is applicable during the optimization steps. Having an efficient approximate sampler for solutions of Min-Cut, it holds that the runtime of the projected SGD algorithm is poly ( 𝑛, 1 /𝜖 ) . We note that during the execution of the algorithm we do not have access to perfectly unbiased samples from mixture of ferromagnetic Ising models. However, we remark that SGD is robust to that inaccuracy in the stochastic oracle. For further details, we refer e.g., to [d'A08]).

G.1.3 Max𝑘 -CSPs

In this problem, we are given a set of variables { 𝑥 𝑢 } 𝑢 ∈𝒰 where |𝒰| = 𝑛 and a set of Boolean predicates 𝑃 . Each variable 𝑥 𝑢 takes values in {-1 , 1 } . Each predicate depends on at most 𝑘 variables. For instance, Max-Cut is a Max-2-CSP. Our goal is to assign values to variables so as to maximize the number of satisfied constraints (i.e., predicates equal to 1). Let us fix a predicate ℎ ∈ 𝑃 , i.e., a Boolean function ℎ : {-1 , 1 } 𝑛 →{ 0 , 1 } which is a 𝑘 -junta. Using standard Fourier analysis, the number of satisfied predicates for the assignment 𝑥 ∈ {-1 , 1 } 𝑛 is

where ̂︀ ℎ 𝑗 ( 𝑆 ) is the Fourier coefficient of the predicate ℎ 𝑗 at 𝑆 .

Theorem 4 (Max𝑘 -CSPs have a Compressed and Efficiently Optimizable Solution Generator) . Consider a prior over Max𝑘 -CSP instances with 𝑛 variables, where 𝑘 ∈ N can be considered constant compared to 𝑛 . For any 𝜖 > 0 , there exists a solution generator 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} such that 𝒫 is complete, compressed with description 𝑂 ( 𝑛 𝑘 ) polylog (1 /𝜖 ) and ℒ + 𝜆𝑅 : 𝒲 ↦→ R is efficiently optimizable via projected stochastic gradient descent in poly ( 𝑛 𝑘 , 1 /𝜖 ) steps for some 𝜆 > 0 .

Proof. Any instance of Max𝑘 -CSP is a list of predicates (i.e., Boolean functions) and our goal is to maximize the number of satisfied predicated with a single assignment 𝑠 ∈ {-1 , 1 } 𝑛 . We show that there exist feature mappings so that the cost of every solution 𝑠 under any instance/predicates list 𝑃 is a bilinear function of the feature vectors (cf. Item 2 of Assumption 1). We consider the order 𝑘 correlation-based feature mappings 𝜓 𝑆 ( 𝑠 ) = ( 𝑠 ⊗ 𝑘 ) ♭ ∈ R 𝑛 𝑘 , where by ( · ) ♭ we denote the flattening operation of the order 𝑘 tensor, and, 𝜓 ℐ ( 𝑃 ) = 𝜓 ℐ ( ℎ 1 , . . . , ℎ | 𝑃 | ) = -∑︀ | 𝑃 | 𝑗 =1 (( ̂︀ ℎ 𝑗 ( 𝑆 )) 𝑆 ⊆ [ 𝑛 ] , | 𝑆 |≤ 𝑘 ) ⊤ ∈ R 𝑛 𝑘 , where ( ̂︀ ℎ 𝑗 ( 𝑆 )) 𝑆 ⊆ [ 𝑛 ] , | 𝑆 |≤ 𝑘 is a vector of size 𝑛 𝑘 with the Fourier coeffients of the 𝑗 -th predicate. We take 𝜓 ℐ being the coordinate-wise sum of these coefficients. The setting the matrix 𝑀 to be the identity matrix 𝐼 ∈ R 𝑛 𝑘 × 𝑛 𝑘 , we get that the cost of any solution 𝑠 can be expressed as the bilinear function 𝜓 ℐ ( 𝑃 ) ⊤ 𝑀𝜓 𝑆 ( 𝑠 ) =

-∑︀ | 𝑃 | 𝑗 =1 ∑︀ 𝑆 ⊆ [ 𝑛 ] , | 𝑆 |≤ 𝑘 ̂︀ ℎ 𝑗 ( 𝑆 ) ∏︀ 𝑢 ∈ 𝑆 𝑥 𝑢 . For any ℎ : {-1 , 1 } 𝑛 →{ 0 , 1 } , we get that the description size of any ̂︀ ℎ ( 𝑆 ) is poly ( 𝑛, 𝑘 ) and so the dimensions of the 𝜓 𝑆 , 𝜓 ℐ feature mappings are polynomial in the description size of ℐ . Moreover, we get that ‖ 𝜓 ℐ ( 𝑃 ) ‖ , ‖ 𝜓 𝑆 ( 𝑠 ) ‖ , ‖ 𝑀 ‖ ≤ poly ( 𝑛 𝑘 ) . Hence, the constants 𝐷 𝑆 , 𝐷 ℐ , 𝐶 are polynomial in the description size of the instance family. Finally, we have that for any 𝑣 , Var 𝑠 ∼ 𝑈 ( 𝑆 ) [ 𝑣 · 𝜓 𝑆 ( 𝑠 )] = Var 𝑠 ∼ 𝑈 ( {-1 , 1 } 𝑛 ) [ 𝑣 · 𝑠 ⊗ 𝑘 ] = Ω( ‖ 𝑣 ‖ 2 2 ) , using Lemma 9, assuming that 𝑣 ∅ is 0 without loss of generality. This implies the result.

G.2 Bipartite Matching and TSP

G.2.1 Maximum Weight Bipartite Matching

In Maximum Weight Bipartite Matching (MWBM) there exists a complete bipartite graph ( 𝐴,𝐵 ) with | 𝐴 | = | 𝐵 | = 𝑛 (the assumptions that the graph is complete and balanced is without loss of generality) with weight matrix 𝑊 where 𝑊 ( 𝑖, 𝑗 ) indicates the value of the edge ( 𝑖, 𝑗 ) , 𝑖 ∈ 𝐴,𝑗 ∈ 𝐵 and the goal is to match the vertices in order to maximize the value. Hence the goal is to maximize 𝐿 (Π) = 𝑊 · Π over all permutation matrices. By the structure of the problem some maximum weight matching is a perfect matching. Furthermore, by negating the weights of the edges we can state the problem as the following minimization problem: given a bipartite graph ( 𝐴,𝐵 ) and weight matrix 𝑊 ∈ ( R ∪ {∞} ) 𝑛 × 𝑛 , find a perfect matching 𝑀 with minimum weight. One of the fundamental results in combinatorial optimization is the polynomial-time blossom algorithm for computing minimum-weight perfect matchings by [Edm65].

We begin this section by showing a variance lower bound under the uniform distribution over the permutation group.

Lemma 10 (Variance Lower Bound) . Let 𝑈 ( S 𝑛 ) be the uniform distribution over 𝑛 × 𝑛 permutation matrices. For any matrix 𝑊 ∈ R 𝑛 × 𝑛 , with ∑︀ 𝑖 𝑊 𝑖𝑗 = 0 and ∑︀ 𝑗 𝑊 𝑖𝑗 = 0 we have

Proof. We have that E Π ∼ 𝑈 ( S 𝑛 ) [Π 𝑖𝑗 ] = 1 /𝑛 and E Π ∼ 𝑈 ( S 𝑛 ) [Π 𝑖𝑗 Π 𝑎𝑏 ] = 1 { 𝑖 = 𝑎 ∧ 𝑗 = 𝑏 } 𝑛 ( 𝑛 -1) + 1 { 𝑖 = 𝑎,𝑗 = 𝑏 } 𝑛 . We have

̸

̸

̸

̸

̸

̸

̸

where to obtain the third equality we used our assumption that ∑︀ 𝑖𝑗 𝑊 𝑖𝑗 = 0 . We observe that, by our assumption that ∑︀ 𝑏 𝑊 𝑎𝑏 = 0 for all 𝑎 it holds ∑︀ 𝑏 = 𝑗 𝑊 𝑎𝑏 = -𝑊 𝑎𝑗 and therefore, we have

̸

̸

̸

̸

̸

Similarly, using the fact that ∑︀ 𝑎 = 𝑖 𝑊 𝑎𝑗 = -𝑊 𝑖𝑗 we obtain that

̸

̸

Therefore, using the above identity, we have that

̸

̸

Combining the above we obtain the claimed identity.

̸

̸

̸

̸

Remark 8. We note that in MWBM the conditions ∑︀ 𝑖 𝑊 𝑖𝑗 = 0 and ∑︀ 𝑗 𝑊 𝑖𝑗 = 0 are without loss of generality.

We next claim that there exists an efficient algorithm for (approximately) sampling such permutation matrices.

Lemma 11 (Efficient Sampling) . There exists an algorithm that generates approximate samples from the Gibbs distribution 𝑝 ( · ; 𝑊 ) with parameter 𝑊 over the symmetric group, i.e., 𝑝 (Π; 𝑊 ) ∝ exp( 𝑊 · Π)1 { Π ∈ S 𝑛 } , in poly ( 𝑛 ) time.

Proof. This lemma essentially requires approximating the permanent of a weighted matrix, since this would imply that one has an approximation of the partition function. Essentially, our goal is to generate a random variable 𝑋 that is 𝜖 -close in statistical distance to the probability measure

Note that the partition function is

where 𝐴 is a non-negative real matrix with entries 𝐴 𝑖𝑗 = exp( 𝑊 𝑖𝑗 ) . Note that in the third equality, we used the isomorphism between permutations and permutation matrices. Hence, 𝑍 ( 𝑊 ) is exactly the permanent of the matrix 𝐴 .

Proposition 10 ([JSV04]) . There exists a fully polynomial randomized approximation scheme for the permanent of an arbitrary 𝑛 × 𝑛 matrix 𝐴 with non-negative entries.

To conclude the proof of the lemma, we need the following standard result.

Proposition 11 (See Appendix H and [Sin12, Jer03]) . For self-reducible problems, fully polynomial approximate integration and fully polynomial approximate sampling are equivalent.

This concludes the proof since weighted matchings are self-reducible (see Appendix H).

The above lemma establishes our goal:

Theorem 5 (MWBM has a Compressed, Efficiently Optimizable and Samplable Solution Generator) . Consider a prior over MWBM instances with 𝑛 nodes. For any 𝜖 > 0 , there exists a solution generator 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} such that 𝒫 is complete, compressed with description poly ( 𝑛 ) polylog (1 /𝜖 ) , ℒ + 𝜆𝑅 : 𝒲 ↦→ R is efficiently optimizable via projected stochastic gradient descent in poly ( 𝑛, 1 /𝜖 ) steps for some 𝜆 > 0 and efficiently samplable in poly ( 𝑛, 1 /𝜖 ) steps.

Proof. Consider an input graph 𝐺 with 𝑛 nodes and adjacency matrix 𝐸 . The feature vector corresponding to a matching can be represented as a binary matrix Π ∈ { 0 , 1 } 𝑛 × 𝑛 with ∑︀ 𝑗 Π 𝑖𝑗 = 1 for all 𝑖 and ∑︀ 𝑖 Π 𝑖𝑗 = 1 for all 𝑗 , i.e., Π is a permutation matrix. Then

Therefore, for a candidate matching 𝑠 , we set 𝜓 𝑆 ( 𝑠 ) to be the matrix Π defined above. Moreover, the feature vector of the graph is the negative (flattened) adjacency matrix -𝐸 ♭ . The cost oracle is then 𝐿 ( 𝑅 ; 𝐸 ) = -∑︀ 𝑖𝑗 𝐸 𝑖𝑗 𝑀 𝑖𝑗 𝑅 𝑖𝑗 perhaps for an unknown weight matrix 𝑀 𝑖𝑗 (see Remark 6). This means that the dimensions of the feature mappings 𝜓 𝑆 , 𝜓 ℐ are polynomial in the bit complexity of ℐ . Moreover, we get that ‖ 𝜓 ℐ ( 𝐼 ) ‖ F , ‖ 𝜓 𝑆 ( 𝑠 ) ‖ F , ‖ 𝑀 ‖ 𝐹 ≤ poly ( 𝑛 ) . We can employ Lemma 10 to get the variance lower bound under the uniform probability distribution in the subspace induced by the matrices satisfying Lemma 10, i.e., the matrices of the parameter space (see Remark 9). Finally, (approximate) sampling from our solution generators can be done efficiently using Lemma 11 and hence (noisy) projected SGD will have a runtime of order poly ( 𝑛, 1 /𝜖 ) (as in the case of MinCut).

We close this section with a remark about Item 3 of Assumption 1.

Remark 9. We note that Item 3 of Assumption 1 can be weakened. We use our variance lower bound in order to handle inner products of the form 𝑤 · 𝑥 where 𝑤 will lie in the parameter space and 𝑥 is the featurization of a solution that lies in some space 𝑋 . Hence it is possible that 𝑤 lies in a low-dimensional subspace of 𝑋 . For our optimization purposes, it suffices to provide variance lower bounds only in the subspace where 𝑤 lies into.

G.2.2 Travelling Salesman Problem

Let us consider a weighted clique 𝐾 𝑛 with 𝑛 vertices and weight matrix 𝑊 ∈ R 𝑛 × 𝑛 . A solution to the TSP instance 𝑊 is a sequence 𝜋 : [ 𝑛 ] → [ 𝑛 ] of the 𝑛 elements indicating the TSP tour ( 𝜋 (1) , 𝜋 (2) , . . . , 𝜋 ( 𝑛 ) , 𝜋 (1)) and suffers a cost

Crucially, the allowed sequences are a proper subset of all possible permutations. For instance, the permutations with fixed points or small cycles are not allowed. In particular, the solution space of TSP corresponds to the set of cyclic permutations with no trivial cycles, i.e., containing an 𝑛 -cycle. Clearly, the number of 𝑛 -cycles is ( 𝑛 -1)! . The goal is to find a tour of minimum cost. Our first goal is to write the cost objective as a linear function of the weight matrix 𝑊 and the feasible solutions, which correspond to cyclic permutations. To this end, we can think of each cyclic permutation 𝜋 as a cyclic permutation matrix Π ∈ { 0 , 1 } 𝑛 × 𝑛 . Then, the desired linearization is given by 𝐿 (Π) = 𝑊 · Π (for a fixed graph instance).

Our next task is to provide a Gibbs measure that generates random cyclic permutations. Let C 𝑛 be the space of 𝑛 × 𝑛 cyclic permutation matrices. Then we have that the tour Π is drawn from

where 𝑊 is the weight matrix. The following key lemma provides guarantees for the performance of our approach to TSP. This lemma allows us to show that the number of optimization steps is poly ( 𝑛, 1 /𝜖 ) .

Lemma 12 (Variance Lower Bound) . Let 𝑈 ( C 𝑛 ) be the uniform distribution over 𝑛 × 𝑛 cyclic permutation matrices. For any matrix 𝑊 ∈ R 𝑛 × 𝑛 , with ∑︀ 𝑖 𝑊 𝑖𝑗 = 0 and ∑︀ 𝑗 𝑊 𝑖𝑗 = 0 we have

Proof. The first step is to compute some standard statistics about cyclic permutations (see Lemma 13). Lemma 13 and the analysis of Lemma 10 gives us that Var Π ∼ 𝑈 ( C 𝑛 ) [ 𝑊 · Π] is equal to

̸

Let us set

We have that

̸

Hence

̸

̸

̸

Due to symmetry, 𝐴 2 = ∑︀ 𝑖,𝑗,𝑎,𝑏 𝑊 𝑖𝑗 𝑊 𝑎𝑏 1 { 𝑗 = 𝑏 = 𝑖 = 𝑎 } = ∑︀ 𝑗 𝑊 2 𝑗𝑗 . It remains to argue about

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

̸

We have that

This gives that

Note that

This implies that

In total, this gives that

̸

̸

̸

̸

̸

̸

̸

̸

̸

Remark 10. We note that in TSP the conditions ∑︀ 𝑖 𝑊 𝑖𝑗 = 0 and ∑︀ 𝑗 𝑊 𝑖𝑗 = 0 are without loss of generality.

The next lemma is a generic lemma that states some properties of random cyclic permutation matrices.

̸

̸

Lemma 13. Consider a uniformly random cyclic permutation matrix Π . Let us fix 𝑖 = 𝑗 and 𝑎 = 𝑏 . Then

E [Π 𝑖𝑗 ] = 1 𝑛 -1 .

̸

̸

̸

̸

̸

̸

̸

̸

̸

Proof. First, note that any matrix that corresponds to a cyclic permutation does not contain fixed points and so the diagonal elements are 0 deterministically. For the first item, the number of cyclic permutations such that 𝑖 → 𝑗 (i.e., Π 𝑖𝑗 = 1 ) is ( 𝑛 -2)! . This implies that the desired expectation is ( 𝑛 -2)! / | C 𝑛 | = 1 / ( 𝑛 -1) . For the second item, if 𝑖 = 𝑎, 𝑗 = 𝑏 , we recover the first item. Otherwise if 𝑖 = 𝑎 or 𝑗 = 𝑏 , then the expectation vanishes since we deal with permutation matrices. Finally, let us consider the case where 𝑖 = 𝑎 and 𝑗 = 𝑏 . Our goal is to count the number of cyclic permutations with 𝑖 → 𝑗 and 𝑎 → 𝑏 .

̸

̸

If 𝑖 = 𝑎 = 𝑗 = 𝑏 = 𝑖 , then there are 𝑛 choices to place 𝑖 and 𝑛 -2 choices to place 𝑎 . Then there are ( 𝑛 -4)! possible orderings for the remaining elements. This gives an expectation equal to 1 / (( 𝑛 -1)( 𝑛 -3)) .

̸

̸

̸

̸

If 𝑖 = 𝑎 = 𝑗 = 𝑏 or 𝑗 = 𝑏 = 𝑖 = 𝑎 , then there are 𝑛 choices for 𝑖 and ( 𝑛 -3)! orderings for the remaining elements. Hence, the expectation is 1 / (( 𝑛 -1)( 𝑛 -2)) .

̸

̸

̸

We note that sampling from our solution generators is the reason that we cannot find an optimal TSP solution efficiently. In general, an algorithm that has converged to an almost optimal parameter 𝑊 ⋆ has to generate samples from the Gibbs measure that is concentrated on cycles with minimum weight. In this low-temperature regime, sampling is NP -hard. We are now ready to state our result.

Theorem 6 (TSP has a Compressed, Efficiently Optimizable Solution Generator) . Consider a prior over TSP instances with 𝑛 nodes. For any 𝜖 > 0 , there exists a solution generator 𝒫 = { 𝑝 ( 𝑤 ) : 𝑤 ∈ 𝒲} such that 𝒫 is complete, compressed with description poly ( 𝑛 ) polylog (1 /𝜖 ) and ℒ + 𝜆𝑅 : 𝒲 ↦→ R is efficiently optimizable via projected stochastic gradient descent in poly ( 𝑛, 1 /𝜖 ) steps for some 𝜆 > 0 .

̸

̸

̸

̸

̸

̸

̸

̸

Proof. Consider an input graph 𝐺 with 𝑛 nodes and weighted adjacency matrix 𝐸 . The feature vector is again a permutation matrix Π with the additional constraint that Π has to represent a single cycle (a tour over all cities). Then

The cost function for TSP is 𝐿 (Π; 𝐸 ) = ∑︀ 𝑖𝑗 𝐸 𝑖𝑗 𝑀 𝑖𝑗 Π 𝑖𝑗 . We refer to Theorem 5 for the details about the feature mappings. We can finally use Lemma 12 to obtain a variance lower bound (in the subspace induced by the parameters satisfying this lemma, see Remark 9) under the uniform measure over the space of cyclic permutation matrices C 𝑛 .

H Sampling and Counting

In this section, we give a quick overview of the connections between approximate sampling and counting. For a formal treatment, we refer to [Sin12].

In what follows, 𝜎 may be thought of as an encoding of an instance of some combinatorial problem, and the 𝜔 of interest are encodings of the structures we wish to generate. Consider a weight function 𝑊 and assume that 𝑊 ( 𝜎, 𝜔 ) is computable in time polynomial in | 𝜎 | .

Definition 4 (Approximate Sampling) . A fully polynomial approximate sampler for (Ω 𝜎 , 𝜋 𝜎 ) is a Probabilistic Turing Machine which, on inputs 𝜎 and 𝜖 ∈ Q + (0 < 𝜖 ≤ 1) , outputs 𝜔 ∈ Σ ⋆ , according to a measure 𝜇 𝜎 satisfying TV( 𝜋 𝜎 , 𝜇 𝜎 ) ≤ 𝜖 , in time bounded by a bivariate polynomial in | 𝜎 | and log(1 /𝜖 ) .

One of the main applications of sampling is to approximate integration. In our setting this means estimating 𝑍 ( 𝜎 ) to some specified relative error.

Definition 5 (Approximate Integration) . A fully polynomial randomized approximation scheme for 𝑍 ( 𝜎 ) is a Probabilistic Turing Machine which on input 𝜎, 𝜖 , outputs an estimate ̂︀ 𝑍 so that

and which runs in time polynomial in | 𝜎 | and 1 /𝜖 .

Definition 6 (Self-Reducible Problems) . An NP search problem is self-reducible if the set of solutions can be partitioned into polynomially many sets each of which is in a one-to-one correspondence with the set of solutions of a smaller instance of the problem, and the polynomial size set of smaller instances are efficiently computable.

For instance, consider the relation MATCH which associates with an undirected graph 𝐺 all matchings (independent sets of edges) of 𝐺 . Then MATCH is self-reducible since, for any edge 𝑒 = ( 𝑢, 𝑣 ) ∈ 𝐸 ( 𝐺 ) , we have that

where 𝐺 1 is the graph obtained by deleting 𝑒 and 𝐺 2 is the graph obtained be deleting both 𝑢 and 𝑣 together with all their incident edges.

Theorem 7 (See Corollary 3.16 in [Sin12]) . For self-reducible problems, approximate integration and good sampling are equivalent.

We remark that the above result holds for the more general class of self-partitionable problems.

I Details of the Experimental Evaluation

We investigate the effect of the entropy regularizer (see Equation (2)) in a very simple setting: we try to find the Max-Cut of a fixed graph 𝐺 , i.e., the support of the prior ℛ is a single graph. We show that while the unregularized objective is often 'stuck' at sub-optimal solutions - and this happens even for very small instances (15 nodes) - of the Max-Cut problem, the regularized version (with the fast/slow mixture scheme) is able to find the optimal solutions. We consider an instance randomly generated by the Erd˝ os-Rényi model 𝐺 ( 𝑛, 𝑝 ) and then optimize the 'vanilla' loss ℒ and the regularized loss ℒ 𝜆 defined in Equation (3). The solutions are vectors 𝑠 ∈ {± 1 } 𝑛 . We first use the feature mapping 𝜓 𝑆 ( 𝑠 ) = ( 𝑠𝑠 ⊤ ) ♭ described in Section 1.1 and an exponential family solution generator that samples a

solution 𝑠 with probability ∝ exp( 𝑤 · 𝜓 𝑆 ( 𝑠 )) for some weight vector 𝑤 ∈ R 𝑛 2 . We also consider optimizing a simple 3-layer ReLU network as solution generator with input 𝑠 ∈ {± 1 } 𝑛 on the same random graphs. We generate 100 random 𝐺 ( 𝑛, 𝑝 ) graphs with 𝑛 = 15 nodes and 𝑝 = 0 . 5 and train solution generators using both the 'vanilla' and the entropy-regularized loss functions. We perform 600 iterations and, for the entropy regularization, we progressively decrease the regularization weight, starting from 10, and dividing it by 2 every 60 iterations. We used a fast/slow mixing with mixture probability 0.2 and inverse temperature rho=0.03 (see Figure 4).

For convenience, we present a pytorch implementation of our simple 3-layer ReLU network here. For more details we refer to our full code submitted in the supplementary material.

Out of the 100 trials we found that our proposed objective was always able to find the optimal cut while the model trained with the vanilla loss was able to find it for approximately 65% of the graphs (for 65 out of 100 using the linear network and for 66 using the ReLU network). In Figure 2 we show two instances where the model trained with the 'vanilla' loss gets stuck on a sub-optimal solution while the entropy-regularized one succeeds in finding the optimal solution. Our experiments show that the regularization term and the fast/slow mixture scheme that we introduced to achieve our main theoretical convergence result, see Section 3 and Proposition 4, are potentially useful for training more realistic models for bigger instances and we leave more extensive experimental evaluation as an interesting direction for future work.

We note that, similarly to our theoretical results, our sampler in this experimental section is of the form 𝑒 score( 𝑠 ; 𝑤 ) , where 𝑠 ∈ {-1 , 1 } 𝑛 ( here 𝑛 is the number of nodes in the graph) is a candidate solution of the Max-Cut problem. The function used is a 3-layer MLP (see Figure Figure 4). Since the instances that we consider here are small ( 𝑛 = 15 ) we can explicitly compute the density (score) of every solution and use that to compute the expected gradient. For larger instances, one could use some approximate sampler (e.g., via Langevin dynamics) to generate samples. The main message of the current experimental section is that even for very small instances of Max-Cut (i.e., with 15 nodes), optimizing the vanilla objective is not sufficient and the iteration gets trapped in local optima. In contrast, our entropy regularized always manages to find the optimal cut.

Peer reviews:
comments: SUMMARY:
The paper presents a thorough consideration on the parametrization of optimizable solution generators that allow using gradient descent to find solutions for combinatorial optimization methods. They find a set of assumptions that enable complete, compressed and efficiently optimizable representations, in particular existence of feature maps of solution and instances onto bounded (norm, diameter and dimensionality) and variance preserving feature spaces which allow for a bilinear cost oracle $c(I,S)=f_I^T M f_S$ for instance/solution features $f_I/f_S$ and a matrix $M$ s.t. $\Vert M \Vert_F\$ is bounded by a constant $C$.

The implications of this result (including the existence of such parametrizations for TSP) are discussed, notably that existance of such generators does not mean P=NP since sampling from the generator might still take exponential time.

The theory is developed on max/min cut, max-k-CSP, max-weight BP matching and TSP and experimentally evalauted on max-cut

STRENGTHS:
Originality: I love this paper for taking the obvious in hindsight question for actually asking what types of representations make combinatorial optimisation via first order methods work on a *principles* basis
Clarity: The paper is superbly written and conveys it's ideas and limitations well
Quality: while I did not have the time to carefully check every brief, on a cursory reading they appear to make sense and align well with intuitions.
Significance: I think this paper will completely change how the deep learning community will think about dealing with combinatorial optimization (or at least I hope so)

WEAKNESSES:
I cannot point to any

QUESTIONS:
None

LIMITATIONS:
I think everything was addressed
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper provides a theoretical analysis of policy-gradient methods for combinatorial optimization problems. It defines a set of three desirable properties one might wish to be fulfilled for such a property, namely being able to generate an approximately optimal solution, being small in size, and enabling efficient optimization via SGD. The main result of the paper is to prove that there exists a policy gradient method satisfying all three properties simultaneously. The proposed method is general enough to capture many well-known combinatorial optimization problems. A small empirical study is provided, too.\n\nSTRENGTHS:\nI think the idea behind this work is great. Theoretical foundations for reinforcement learning methods applied to CO problems are highly needed. Defining a set of desirable properties and analyzing whether and how they can be satisfied simultaneously seems to be the right approach.\n\nWEAKNESSES:\nSince I believe that the contributions of this paper are great, I feel really sorry that I cannot give a more positive evaluation at the moment. I try to give as much constructive feedback as possible and encourage the authors to revise the manuscript and, if not accepted here, resubmit to another top venue. My main concern is:\n- I have a few mathematical issues / questions, where I do not understand the paper. I think one source of these issues is that the model of computation is not well-defined. The authors wildly switch between bit-representations and real models of computation. To solve these issues, the authors should clearly define in which model of computation they work and make all assumptions / requirements / statements consistent with this model. There are a few more unrelated issues, see my more specific questions in the \"Questions\" section.\n\nApart from that, I have a whole bunch of secondary comments, see below:\n- I think the paper (implicitly and sometimes explicitly) oversells the contribution of [BPL+16] to amplify the motivation for their own work. So far, reinforcement learning is NOT a state-of-the-art method for combinatorial optimization problems like TSP. The work of [BPL+16] is great as a proof of concept, but statements like \"[BPL+16] [...] generate very good solutions for (Euclidean) TSP instances\" without mentioning that these are very small toy problems should be avoided. I think the fact that NNs for CO are still in their beginnings should be pointed out more clearly and the motivation of this paper should build upon a larger variety of prior work than only [BPL+16].\n- line 103: I suppose you want to refer to Def. 2 and not to Qu. 1 here (in particular, because otherwise you refer to Qu. 1 before even stating it).\n- line 143: I find it more natural to write this with $L(s,I)$ instead of $\\mathcal{O}(s,I)$. I understand that this is the same by definition, but here the focus is more on the cost structure and less on the fact that such an oracle exists. Maybe one could even drop the notation $\\mathcal{O}(s,I)$ everywhere in the paper and use always $L$? In Definition 1, one could instead just write in words that such an oracle exists.\n- line 196: I feel that the split into supervised and unsupervised models here is very artificial because papers in both categories apply sophisticated algorithms mixing very different paradigms. I doubt that the negative result [YGS20] is directly applicable to the settings of the three papers you cite for the supervised case. I suggest not to artificially split the related work into these two categories.\n- Related work: even though of a very different flavor, I think the following two papers about the theoretical ability of neural networks to solve CO problems should be discussed in the related work section: Hertrich, C., & Skutella, M. (2023). Provably good solutions to the knapsack problem via neural networks of bounded size. INFORMS Journal on Computing. AND: Hertrich, C., & Sering, L. (2023). ReLU neural networks of polynomial size for exact maximum flow computation. In International Conference on Integer Programming and Combinatorial Optimization.\n- line 219: please either provide a proof or a reference for the reformulation of the MaxCut problem via the Laplacian.\n- line 241: say that Thm. 4 is in the appendix.\n- line 246: a unknown -> an unknown. Also the whole sentence is a bit hard to read, maybe split into two?\n- line 286: \"the point $\\bar{W}=-\\tau M$ when $\\tau\\rightarrow+\\infty$\" is not a point. What you write here is not mathematically precise, even though I understand what you mean. Consider revising.\n- lines 323/324: I recommend a comma between \"fast\" and \"making\".\n- line 344: there is a \"to\" too much (between \"we\" and \"pick\").\n- line 382: you should not assume that everyone knows what $G(n,p)$ is.\n\nQUESTIONS:\n- line 73: why is this the gradient? Please provide some explanation. In particular: where does the logarithm come from?\n- line 75: what do you mean by \"using only access to a solution cost oracle\"? Do you want to point out that something particular is not used? If so, what? I suppose you also need to be able to sample $I$ and $s$ according to their respective distributions, and you need to be able to compute the gradient of $log(p(s;I;w))$ w.r.t. $w$.\n- line 96: what is the \"description size of the parameter space $\\mathcal{W}$\"? Is it the number of bits required to represent any parameter $w\\in \\mathcal{W}$? But then, $\\mathcal{W}$ is a finite set, so how can you ever perform gradient descent on it?\n- line 99: why do you allow running time polynomial in $1/\\epsilon$, but the parameter space must be polynomial in $\\log(1/\\epsilon)$?\n- line 107: I do not understand what \"the full parametrization of all distributions over the hypercube\" is. There are uncountably many such distributions, so how can you ever represent them in a set $\\mathcal{W}$ of finite description size?\n- Remark 5: Is this a formal statement or an intuitive statement? I really would like to see more details about how exactly the cited paper implies a negative result for neural network solution samplers in the setting of your paper.\n- Remark 6: What if I WANT to encode the weights into the instances? I suppose you would it call \"known\" weights then. But it is less clear how to define the feature maps then and still preserve bilinearity. I do not see why what you call \"unknown\" weights is the more general / difficult case.\n\nLIMITATIONS:\nIn principal, the authors provide all details required to understand the scope of their results. The discussion on limitations could be improved by answering the following two questions:\n- Are there natural combinatorial optimization problems to which the framework is not applicable? If so, which problems, and why?\n- It seems like the proposed method is theoretically superior to what people use in practice. Is this true? And if so, why is it not the standard method in practical settings nowadays? Do you expect it to become that?", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a theoretical framework for analyzing the effectiveness of deep models trained by gradient-based methods as solution generators for combinatorial problems. The authors first investigate the complete, compressed and efficiently optimizable properties of the solution generator on many combinatorial tasks. To address the challenges of minimizers at infinity and vanish gradients, the authors devise an entropy regularization and a fast/slow mixture generation scheme. Experiments demonstrate that the proposed method helps address vanishing-gradient issues and escape bad stationary points.\n\nSTRENGTHS:\n1. The paper gives a positive answer to the existence of complete, compressed and efficiently optimizable solution generators for combinatorial tasks, and it designs a family of such solution generators.\n2. The paper addresses the challenges of minimizers at infinity and vanishing gradients by the proposed entropy regularization and fast/slow mixture generation scheme.\n3. A general and solid foundational theorem (Theorem 1) is clearly presented.\n\nWEAKNESSES:\n1. The authors may want to conduct experiments on more combinatorial problems with larger scales to demonstrate the effectiveness of the proposed method.\n2. The authors analyze the existence and properties of the feature mappings for the solutions and instances, but how to learn such mappings remains unexplored.\n3. The authors apply MLP to combinatorial problems with a fixed number of graph nodes. However, MLPs fail to process instances with varying scales. The authors may want to consider more suitable models such as the graph neural network (GNN).\n\nQUESTIONS:\n1. Could you please give some examples of combinatorial problems that do not satisfy Assumption 1?\n2. Could you please provide more details on the representations of input data, such as the input features and data format?\n\nLIMITATIONS:\nAssumption 1 might be difficult to check for a general combinatorial problem, which may limit the applicability of the theoretical results in this paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper deals with a significant question at the intersection of combinatorial optimization and continuous-based optimization: is it possible to design solution generators for combinatorial problems that are (1) expressive enough to generate approximately optimal solutions, (2) tractable so that their parameterization is only polynomial in the number of inputs, and (2) efficiently optimizable so that only a polynomial number of stochastic gradient descent (SGD) steps is required to learn a parameterization with almost optimal performance? The paper answers this question in the affirmative. It develops a general framework that can be used to describe complete, compressed and efficiently optimizable solution generators, which can be instantiated to accommodate a wide range of NP-hard problems. The framework requires feature mappings for both the instances and the solutions tat have a number of desired properties. Assuming this is the case, the authors provide a very general positive result. The paper highlights two challenges related to the optimization. First, the loss function may accept a minimizer at infinity, so that gradient descent may get stuck to sub-optimal configurations. The authors address this by introducing an entropy regularization term. Second, vanishing gradients may slow down optimization, rendering it inefficient. This is addressed by introducing a mixture of a fast and a slow solution generator: the fast component helps to reach the optimal solution while the slow component helps to keep a non-zero variance throughout the optimization process, hence avoiding the problem of vanishing gradients. The authors provide experimental evidence in favor of their entropy regularizer and the fast/slow mixing scheme.\n\nSTRENGTHS:\n1. The paper is extremely well-written. The authors have done an excellent job presenting in a simple yet rigorous manner the main insights behind their framework and the various design choices. Every single choice is adequately justified, which makes it very easy even for non-familiar readers to grasp the main messages of this work. I also liked the the fact that the authors took extra care to elucidate some potentially confusing aspects, e.g., with respect to the P vs. NP question.\n2. The main result requires very reasonable assumptions and is general enough to encompass several standard hard problems. \n3. The two proposed tricks (entropy regularization and fast/slow mixing scheme) are extremely well motivated but also justified theoretically. The ablation study at the end confirms their usefulness empirically.\n4. The framework is novel and a major part of its novelty precisely stems from its generality. \n5. Even though I was not able to check all math details, the part that I was able to check was free of problems.\n\nWEAKNESSES:\n1. I understand that the main focus of this work is to provide a generic framework with reasonable assumptions and strong performance guarantees. I feel that the authors have delivered on that premise. That said, readers may be left with the question: what is the true empirical potential of this framework, especially when we compare it to state of the art combinatorial neural solvers? After reading the paper, I was not really clear whether this work is mainly about introducing a generic framework, or if it was also about proving its practical value on various benchmarks.\n\nQUESTIONS:\n1. Is this work only intended to theoreticians or to practitioners as well? If the latter is the case, then the authors would need to provide evidence for that claim, e.g., performance of their solution generators compared to other continuous solvers. I feel that this question is not answered in the current paper.\n2. On a similar note, the number of nodes n=15 used in the ablation study is small - could these methods scale to larger instances? Do the authors have any take-away messages for practitioners who may be interested in trying out such methods? Is it perhaps fair to say that the proposed framework mostly serves as an abstraction but without necessarily powerful practical implications?\n3. In the discussion of P vs. NP, the authors explain that sampling from their solution generators may be computationally expensive but add that technique based on Langevin dynamics may help to circumvent this issue in practice. It would be worthwhile for the authors to see whether such techniques would be of any help, especially with large instances.\n\nLIMITATIONS:\nNone.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}