Annealed Training for Combinatorial Optimization on Graphs
Abstract
Learning neural networks for CO problems is notoriously difficult given the lack of labeled data as the training gets trapped easily at local optima.
However, the hardness of combinatorial optimization (CO) problems hinders collecting solutions for supervised learning.
We propose a simple but effective unsupervised annealed training framework for CO problems in this work.
In particular, we transform CO problems into unbiased energy-based models (EBMs).
We carefully selected the penalties terms to make the EBMs as smooth as possible.
Then we train graph neural networks to approximate the EBMs and we introduce an annealed loss function to prevent the training from being stuck at local optima near the initialization.
An experimental evaluation demonstrates that our annealed training framework obtains substantial improvements.
In four types of CO problems, our method achieves performance substantially better than other unsupervised neural methods on both synthetic and real-world graphs.
1. Introduction
Combinatorial Optimization (CO) problems occur whenever there is a requirement to select the best option from a finite set of alternatives.
They arise in various application areas, like business, medicine, and engineering [37].
Many CO problems are NP-complete [16, 25].
Thus, excluding the use of exact algorithms to find the optimal solution [36, 56], different heuristic methods are employed to find suitable solutions in a reasonable time [14, 21, 27, 34].
Often, instances from the same combinatorial optimization problem family are solved repeatedly, giving rise to the opportunity for learning to improve the heuristic [7].
Recently, learning algorithms for CO problems has shown much promise, including supervised [17, 26, 30, 33, 42], unsupervised [24, 46] and reinforcement learning [9, 11, 44, 62] The success of supervised learning relies on labeled data.
However, solving a hard problem could take several hours or even days and is computationally prohibitive [61].
Reinforcement learning, suffering from its larger state space and lack of full differentiability, tends to be more challenging and time-consuming to train.
Unsupervised learning usually transforms a CO problem into an optimization problem with a differentiable objective function f where the minima represent discrete solutions [21, 24, 43].
Although this framework allows for efficient learning on large, unlabeled datasets, it is not without challenges.
The objective function is typically highly non-convex [31].
During learning, the model’s parameters can easily get trapped near a local optimum close to the initialization, never reaching the optimal set of parameters.
This makes unsupervised learning for CO problems extremely hard.
To address this challenge, we propose an annealed training framework.
In detail, given a
CO problem, we consider a tempered EBM P
τ ∝ e − f ( x ) /τ , where the energy function f unifies constrained or unconstrained CO problems via the big-M method, that is to say, adding large penalties for violated constraints.
We derive the minimum values of the penalty coefficient in different CO problems that give us the smoothest, unbiased energy-based models.
We train a graph neural network (GNN) that predicts a variational distribution Q ϕ to approximate the energy-based model P τ .
During training, we set a high initial temperature τ and decrease it gradually during the training process.
When τ is large, P τ is close to a uniform distribution and only has shallow local optima, such that the parameter θ can traverse to distant regions.
When τ decreases to values small enough, the unbiased model P τ will concentrate on the optimal solutions to the original CO problem.
The experiments are evaluated on four NP-hard graph CO problems: MIS, maximum clique,
MDS, and minimum cut.
On both synthetic and real-world graphs, our annealed training framework achieves excellent performance compared to other unsupervised neural methods [24, 46], classical algorithms [1, 8], and integer solvers [18].
The ablation study demonstrates the importance of selecting proper penalty coefficients and cooling schedules.
In summary, our work has the following contributions:
• We propose an annealed learning framework for generic unsupervised learning on combinatorial optimization problems in Sec.
It is simple to implement yet effective in improving unsupervised learning across various problems on both synthetic and real graphs (Sec. 3).
• We conducted ablation studies in Sec. 4 that show: 1) annealed training enables the parameters to escape from local optima and traverse a longer distance, 2) selecting proper penalty coefficients is essential, 3) Using initial temperature large enough is critical.
2. Annealed Training for Combinatorial Optimization
We want to learn a graph neural network G θ to solve combinatorial optimization problems.
Given an instance I , the G θ generates a feature ϕ
= G θ ( I ) that determines a variational distribution Q ϕ , from which we decode solutions.
This section presents our annealed training framework for training
G θ .
We first represent CO problems via an energy-based model.
Then, we define the annealed loss function and explain how it helps in training.
We give a toy example in Sec.
2.4 to help the understanding.
2.1. Energy Based Model
We denote the set of combinatorial optimization (CO) problems as I .
An instance I ∈
I is
[Equation]
where c ( · ) is the objective function and ψ i ∈ { 0 , 1 } indicates if the i-th constraint is satisfied.
We rewrite the constrained problem into an equivalent unconstrained form via the big M method:
[Equation]
If f ( I ) has its smallest values on optimal solutions for (1), we refer it to unbiased.
The selection of penalty coefficient β plays an important role in the success of training, and we will discuss our choice of β detailedly in section C. Using unbiased f ( I ) as an energy to measure the fitness of a solution x , solving CO problems is converted to finding low energy states.
Accordingly, we can define the unbiased energy-based models (EBMs):
[Equation]
where a state x is more likely being observed than another state x ′ if it has a lower energy
f ( I ) ( x ) <
f ( I ) ( x ′ ) .
The EBMs naturally introduce a temperature τ to control the smoothness of the distribution.
When f is unbiased, it has the following property:
Proposition 1 Assume f is unbiased, that’s to say, all minimizers of (2) are feasible solutions for (1) .
When the temperature τ increases to infinity, the energy-based model P τ converges to a uniform distribution over the whole state space { 0 , 1 } n .
When the temperature τ decreases to zero, the energy-based model P τ converges to a uniform distribution over the optimal solutions for (1) .
The proposition above shows that the temperature τ in unbiased EBMs provides an interpolation between a flat uniform distribution and a sharp distribution concentrated on optimal solutions.
This idea is the key to the success of simulated annealing [27] in inference tasks.
We will show that the temperature also helps in learning.
2.2. Tempered Loss and Parameterization
We want to learn a graph neural network G θ parameterized by θ .
Given an instance I ∈ I ,
G θ ( I ) =
ϕ generates a vector ϕ that determines a variational distribution Q ϕ to approximate the target distribution P ( I ) τ .
We want to minimize the KL-divergence:
[Equation]
[Equation]
where H ( p ) = − (cid:80) x p ( x ) log p ( x ) denote the entropy of a distribution p .
Removing the terms not involving ϕ and multiplying the constant τ , we define our annealed loss functions for ϕ and τ as:
[Equation]
In this work, we consider the variational distribution as a product distribution:
[Equation]
where ϕ ∈ [0 , 1] n .
Such a form is popular in learning graphical neural networks for combinatorial optimization [12, 24, 30] for its simplicity and effectiveness.
However, directly applying it to unsupervised learning is challenging.
Unlike supervised learning, where the loss function crossentropy is convex for ϕ , L τ ( ϕ, I ) in unsupervised learning could be highly non-convex, especially when τ is small.
2.3. Annealed Training
To address the non-convexity in training, we employ annealed training.
In particular, we use a large initial temperature τ 0 to smooth the loss function and reduce τ t gradually to zero during training.
From proposition 1, it can be seen as a curriculum learning [6] along the interpolation path from the easier uniform distribution to a more challenging target distribution.
Why is it helpful?
We need a thorough investigation of the training procedure to answer this.
Since the loss function (7) is the expectation over the set of instances I , we use a batch of instances
I 1 , ..., I B to calculate the empirical loss ˆ L τ ( θ ) and perform stochastic gradient descent.
It gives:
[Equation]
In (10), we assume the batch introduces a stochastic term ξ in gradient w.r.t.
θ .
In (11), we incorporate the stochastic term into the gradient with respect to ϕ .
When we assume ζ is a Gaussian noise, the inner term g = ∇ ϕ L τ ( ϕ, I ) | ϕ
= G θ ( I ) + ζ performs as a stochastic Langevin gradient with respect to ϕ [55].
Since the training data is sampled from a fixed distribution I ∼ I , the scale of the noise ζ is also fixed.
When L τ ( ϕ, i ) is unsmooth, the randomness from ζ is negligible compared to the gradient ∇ L τ ( ϕ, i ) and can not bring ϕ out of local optima.
By introducing the temperate τ , we smooth the loss function and reduce the magnitude of ∇ L τ ( ϕ, i ) .
During the training, the annealed training performs an implicit simulated annealing [27] for ϕ .
2.4. A Toy Example
We look at a toy example to have a more intuitive understanding of the annealed training.
Consider a
MIS problem on an undirected, unweighted graph G =
( V, E ) , the corresponding energy function f ( x ) is:
[Equation]
Its correctness can be justified by proposition 2.
When we use the variational distribution Q ϕ in (8), the first term in L τ ( ϕ, I ) becomes to:
[Equation]
and accordingly, the gradient w.r.t.
ϕ is:
[Equation]
where we assume ζ ∼ N (0 , σ 2 ) for a very small σ .
When the temperature τ = 0 , ϕ i will collapse to either 0 or 1 very fast.
When ϕ i = 1 , we have g = − 1+ ζ , when ϕ
i = 0 , we have g ≥ 1+ ζ .
Since σ is small, the noise ζ can hardly have an effect, and ϕ will be stuck at local optima, i.e., any maximal independent set such as figure.
1 (a).
In figure.
1, we simulate the input (a) at decreasing temperatures τ = 1 . 0 , 0 . 5 , 0 .
When τ is large, all ϕ i will be pushed to a neutral state, e.g., in the figure.
1 (b) where the difference of ϕ i is at scale 10 − 3 .
In this case, the noise ζ can significantly affect the sign of the gradient g and lead to phase transitions.
By gradually decreasing the temperature, ϕ collapses to the global optimum and provides correct guidance to update θ .
0   ( a ) ( b ) ( c ) ( d )
Input τ = 1.
τ = 0.1 τ = 0.
[Table]
3. Experiments 3.1. Results
We report the results for MIS in Table 1.
We also the results for maximum clique, MDS, and minimum cut in Table 2, Table 3, Table 4 in Appendix E.1, respectively.
More results for comparison with supervised learning methods and evaluation on very large graphs are provided in E.2 and E.3.
In the MIS and maximum clique, we report the ratios computed by dividing the optimal value by the obtained value (the larger, the better).
In the MDS, we report the ratios computed from the obtained value by dividing optimal value (the larger, the better).
Our method outperforms greedy heuristics, classical algorithms, and other neural based methods.
Besides, with annealed training, the learned
GNN outperforms MFA in most problems, with less number of iterations.
It indicates that learning the shared patterns in graphs is helpful in solving CO problems.
Comparing to integer solver, Gurobi is able to obtain good ratio on smaller graphs.
On larger scale instances, our method can achieve comparable or even better results.
3.2. Parameter Change Distance
We want to stress that we use the same graph neural network as Erdos or RUNCSP, and the performance improvements come from our annealed training framework.
In scatter plot 2, 3, we report the relative change for the parameters of GNN in MIS and MDS problems on the Twitter dataset.
The relative change is calculated as ∥ u − v ∥ 2 ∥ v ∥ 2 , where v and u are vectors flattened from the parameters of GNN before and after training.
For each method, we run 20 seeds.
After introducing the annealed training, we see that both the ratio and the relative change of the parameters have a systematic increase, meaning the parameters of GNN can traverse to more distant regions and
[Figure]
[Figure] [Figure]
find better optima in annealed learning.
We believe this effectively supports that annealed training prevents the training from being stuck at local optima.
4. Ablation Study
We conduct an ablation study on the MDS problem on the small BA graphs to answer two questions:
How does the penalty coefficient β in (2) influence the performance?
How does the annealing schedule influence the performance?
Penalty Coefficient
In the MDS problem
[Figure]
we know that the minimum penalty coefficient β needed to ensure the EBMs unbiased on the unweighted BA graphs is β = 1 . 0 .
To justify the importance to use the minimum penalty, we evaluate the performance for β = {0.0, 0.25, 0.5, 0.75, 1.0, 2.0, 3.0, 5.0}.
For each β , we run experiments with five random seeds
[Figure]
and we report the result in Figure 4.
We can see that the minimum penalty β
= 1 has the best ratio.
When the penalty coefficient β < 1 , the EBMs (3) are biased and have weights on infeasible solutions, thereby reducing the performance.
When the penalty coefficient β > 1 , the energy model (3) becomes less smooth and increases the difficulty in training.
The penalty coefficient β = 1 gives the smoothest unbiased EBMs and has the best performance.
We want to note that when β = 0 , the loss function is non-informative, and the performance ratio can be as low as 0 .
3 , so we do not plot its result in the figure.
4.2. Annealing Schedule
We report the results in Figure 5 in Appendix E.4 for different annealing schedule.
We see that the performance is robust for whichever convex, linear, or concave schedule is used.
The more important factor is the initial temperature τ 0 .
The performance is reduced when τ 0 is too small as the energy-based model (3) is not smooth enough, and the performance is robust when τ 0 is large.
Appendix A. Types of Problems Solvable
The current method uses conditional decoding [39] to sample solutions from the learned variational distributions, which requires monotonic post-processing to make sure the final solution is feasible.
For example, in the maximum independent set, the monotonic post-processing is removing nodes when conflict happens, in the minimum dominant set, the monotonic post-processing is adding nodes when a node has not been covered.
Such a framework can be applied to CO problems that have trivial solutions, such as set covering problems, but can not be applied to CO problems with complicated constraints, such as vehicle routing problems.
Appendix B. Related Work
Recently, there has been a surge of interest in learning algorithms for CO problems [7].
Supervised learning is widely used.
Numerous works have combined GNNs with search procedures to solve classical CO problems, such as the traveling salesman problem [23, 38, 51], graph matching [53, 54], quadratic assignments [35], graph coloring [29], and MIS [30].
Another fruitful direction is combining learning with existing solvers.
For example, in the branch and bound algorithm, Gasse et al.
[17], He et al.
[19], Khalil et al.
[26], Nair et al.
[33] learn the variable selection policy by imitating the decision of oracle or rules designed by human experts.
However, the success of supervised learning relies on large labeled datasets, which is hard to efficiently generate in an unbiased and representative manner [61].
Many works, therefore, choose to use reinforcement learning instead.
Dai et al.
[11] combines
Q-learning with greedy algorithms to solve CO problems on graphs.
Q-learning is also used in [4] for maximum subgraph problem.
Sun et al.
[44] uses an evolutionary strategy to learn variable selection in the branch and bound algorithm.
Yolcu and Póczos [62] employs REINFORCE algorithm to learn local heuristics for SAT problems.
Chen and Tian [9] uses actor-critic learning to learn a local rewriting algorithm.
Despite being a promising approach that avoids using labeled data, reinforcement learning is typically sample inefficient and notoriously unstable to train due to poor gradient estimations, correlations present in the sequence of observations, and hard explorations [15, 45].
Works in unsupervised learning show promising results.
In initial attempts, Hopfield and Tank [21], Ramanujam and Sadayappan [40], Van den Bout and Miller [48] transform CO problems into optimization problem of neural networks with differentible objective functions.
More recently, a series of deep learning approaches emerges.
Yao et al.
[60] train GNN for the max-cut problem by optimizing a relaxation of the cut objective, Toenshoff et al.
[46] trains RNN for maximum-SAT via maximizing the probability of its prediction.
Karalias and Loukas [24] use a GNN to predict the distribution and the graphical neural network to minimize the expectation of the objective function on this distribution.
The probabilistic method provides a good framework for unsupervised learning.
However, optimizing the distribution is typically non-convex [31], making the training very unstable.
Appendix C. Case Study
We consider four combinatorial optimization problems on graphs in this work: maximum independent set (MIS), maximum clique, minimum dominate set (MDS), and minimum cut.
An undirected weighted graph can represent all problems G =
( V, E, w ) , where V = { 1 , ..., n } is the set of nodes,
E is the set of edges, and w is the weight function.
For any i ∈ V , w i = w ( i ) is the weight of the node.
For any ( i, j ) ∈ E , w ij = w ( i, j ) is the weight of the edge.
For each problem, we derive the minimum value of the penalty coefficient β
such that the energy function has the lowest energy at optimal solutions, and we use the derived values to design the loss functions in our experiments.
C.1. Maximum Independent Set
In MIS, we use the energy function:
[Equation]
We are going to prove the following proposition.
Proposition 2 If β ij ≥ min { w i , w j } for all ( i, j ) ∈ E , then for any x ∈ { 0 , 1 } n , there exists a x ′ ∈ { 0 , 1 } n that satisfies the constraints in (15) and has lower energy:
f ( x ′ ) ≤ f ( x ) .
Proof For arbitrary x ∈ { 0 , 1 } n , if x satisfies all constraints, we only need to let x ′ = x .
Else, there must exist an edge ( i, j ) ∈ E , such that x i x j = 1 .
Denote k = arg min { w i , w j } , we define x ′ i =
x i if i ̸
= k and x ′ k
= 0 .
In this case, we have:
[Equation]
Thus we show f ( x ′ ) ≤
f ( x ) .
On the other side, consider a graph G =
( V = { 1 , 2 } , E = { (1 , 2) } ) and β 12 < w 1 < w 2 .
Then the maximum independent set is { 2 } , which can be represented by x = (0 , 1) .
However, in this case, let x ′ =
(1 , 1) is feasible while f ( x ′ ) ≤ f ( x ) .
This means the condition we just derived is sharp.
C.2. Maximum Clique
A clique is a subset of the vertices S ⊆ V , such that every two distinct i, j ∈ S are adjacent: ( i, j ) ∈ E .
The maximum problem is finding a clique S with the largest weight.
Rigorously, if we denote x i = 1 to indicate i ∈ S and x i = 0 to indicate i / ∈ S , the problem can be formulated as:
[Equation]
where E c = { ( i, j ) ∈
V × V : i ̸
= j, ( i, j ) / ∈ E } is the set of complement edges on graph G .
We define the corresponding energy function:
[Equation]
We are going to prove the following proposition.
Proposition
If β ij ≥ min { w i , w j } for all ( i, j ) ∈ E c , then for any x ∈ { 0 , 1 } n , there exists a x ′ ∈ { 0 , 1 } n that satisfies the constraints in (17) and has lower energy:
f ( x ′ ) ≤ f ( x ) .
Proof For arbitrary x ∈ { 0 , 1 } n , if x satisfies all constraints, we only need to let x ′ = x .
Else, there must exist an edge ( i,
j ) ∈
E c , such that x i x j = 1 .
Denote k = arg min { w i , w j } , we define x ′ i =
x i if i ̸
= k and x ′ k
= 0 .
In this case, we have:
[Equation]
Thus we show f ( x ′ ) ≤
f ( x ) .
On the other side, consider a graph G =
( V = { 1 , 2 } , E = {} ) and β 12 < w 1 < w 2 .
Then the maximum clique is { 2 } , which can be represented by x = (0 , 1) .
However, in this case, let x ′ =
(1 , 1) is feasible while f ( x ′ ) ≤ f ( x ) .
This means the condition we just derived is sharp.
C.3. Minimum Dominate Set
In MIS, we use the energy function:
[Equation]
We are going to prove the following proposition.
Proposition
If β i ≥ min k { w k : k ∈ N ( i ) or k = i } , then for any x ∈ { 0 , 1 } n , there exists a x ′ ∈ { 0 , 1 } n that satisfies the constraints in (17) and has lower energy:
f ( x ′ ) ≤ f ( x ) .
Proof For arbitrary x ∈ { 0 , 1 } n , if x satisfies all constraints, we only need to let x ′ = x .
Else, there must exist a node t ∈ V , such that x t = 0 and x j = 0 for all j ∈ N ( t ) .
Let k = arg min { w j : j ∈
N ( t ) , or j = t } , we define x ′ i
= x i if i ̸
= k and x ′ k
= 1 .
In this case, we have:
[Equation]
Thus, we prove f ( x ′ ) ≤ f ( x ) .
On the other side, consider a graph G =
( V = { 1 } , E = {} ) and β 1 < w 1 .
Then the maximum clique is { 1 } , which can be represented by x = (1) .
However, in this case, let x ′ =
(0) is feasible while f ( x ′ ) ≤ f ( x ) .
This means the condition we just derived is sharp.
C.4. Minimum Cut
In MIS, we use the energy function:
[Equation]
We are going to prove the following proposition.
Proposition 5 If β ≥ max i { (cid:80) j ∈ N ( i ) | w i,j |} , then any x ∈ { 0 , 1 } n , there exists a x ′ ∈ { 0 , 1 } n that satisfies the constraints in (17) and has lower energy:
f ( x ′ ) ≤ f ( x ) .
Appendix D. Experiment Details D.1. Hardware
All methods were run on Intel(R)
Xeon(R)
Gold 5215 CPU @
2.50GHz, with 377GB of available
RAM.
The neural networks were executed on a single RTX6000 25GB graphics card.
The code was executed on version 1.9.0 of PyTorch and version 1.7.2 of PyTorch Geometric.
D.2. Settings
Dataset : For MIS and maximum clique, problems on both real and random graphs are easy [12].
Hence, we follow Karalias and Loukas [24] to use RB graphs [57], designed to generate hard instances.
We use a small dataset containing graphs with 200-300 nodes and a large dataset containing graphs with 800-1200 nodes.
For MDS, we follow Dai et al.
[12] to use BA graphs with 4 attaching edges [5].
We also use a small dataset containing graphs with 200-300 nodes and a large dataset containing graphs with 800-1200 nodes.
We also use real graph datasets Collab, Twitter from TUdataset [32].
For minimum cut, we follow Karalias and Loukas [24] and use real graph datasets including SF- [59], Facebook [47], and Twitter [32].
For RB graphs, the optimal solution is known during the graph construction.
For other problems, we generate the "ground truth" solution through Gurobi 9.
[18] with a time limit of 3600 seconds.
For synthetic datasets, we generate 2000 graphs for training, 500 for validation, and 500 for testing.
For real datasets, we follow Karalias and Loukas [24] and use a 60-20-20 split for training, validating, and testing.
Implementation : We train our graph neural network on training data with 500 epochs.
We choose the penalty coefficient β at the critical point for each problem type.
We use the schedule:
[Equation]
23) where τ 0 is chosen as the Lipschitz constant of the energy function (2) and α is selected to make sure the final temperature τ 500 = 0 .
Since the contribution of this work focuses on the training framework, the architecture of the graph neural network is not important.
Hence, we provide results from applying annealing training to Karalias and Loukas [24] and Toenshoff et al.
[46] for fair comparison, denoted as "Annealed Erdos" and "Annealed RUNCSP" respectively.
In particular, the architecture from Karalias and Loukas [24] consists of multiple layers of the Graph Isomorphism
Network [58] and a graph Attention [50].
More details refer to Karalias and Loukas [24].
Moreover, the architecture from Toenshoff et al.
[46] creates a network that approximates a Constraint Language [13] using a message-passing GNN using an LSTM for internal updates [20].
With both of these
GNN architectures, after obtaining the variational distribution Q ϕ (8), we generate the solution via conditional decoding [39].
Baselines : We compare our method with unsupervised neural methods, classical algorithms, and integer programming solvers.
To establish a strong baseline for neural methods, we use the Erdos
GNN [24], the state-of-the-art unsupervised learning framework for combinatorial optimization problems.
For maximum clique and MIS, we transform the problem to constraint programming and compare them with RUNCSP [46].
We also implement the annealed training version of RUNCSP and denote it as RUNCSP(A).
We followed Karalias and Loukas [24] for minimum cut and built the L
GNN and L2 GNN.
In classical algorithms, we consider greedy algorithms and mean field annealing (MFA) [8].
MFA also runs mean field approximation [3] to predict a variational distribution as our method.
The difference is that the update rule of MFA is determined after seeing the current graph, while the parameters in GNN are trained on the whole dataset.
Also, in minimum cut, we follow [24] to compare with well-known and advanced algorithms: Pageran-Nibble [2], Capacity Releasing
Diffusion (CRD) [52], Max-flow Quotient-cut Improvement [28], and Simple-Local [49].
For integer programming solver, we use Gurobi 9.0 [18] and set different time limits t .
We denote G( t s) as
Gurobi 9.0 ( t s).
where t is the solving time limit.
One needs to notice that Gurobi has proprocessing before solving, so the actual running time can be longer than the given time limit.
D.3. Greedy Algorithm
For MIS, greedy algorithm can be described in the following steps:
Pick the variable i has the smallest degree d i in the candidate set.
Delete i and all its neighborhood N ( i )
=
{ j : ( i, j ) ∈ E } on the current graph.
Repeat step 1-2, until the current graph is empty.
For maximum clique, we first transform the graph into its complementary, then apply the greedy algorithm for MIS.
For MDS, greedy algorithm can be described in the following steps:
For every node i , initialize its state s i = 1 to indicate whether it has not been covered.
For every node i , initialize its covering number c i
=
s i + (cid:80)
j ∈ N ( i ) s j to indicate how many nodes can be covered by selecting node i 3.
Select the node i has the largest covering number c i . 4.
Mark s i = 0 , and s j = 0 for j ∈ N ( i ) .
Repeat step 3-4, until all s i = 0 .
D.4. Datasets
For MIS and maximum clique, we follow Karalias and Loukas [24] and use RB graphs [57].
The construction of RB graphs has 4 parameters n, k, p .
Following Karalias and Loukas [24], for small graphs, we use n uniformly sampled from the integers [20 , 25] and k uniformly sampled from [5 , 12] ; for large graphs, we use n uniformly sampled from the integers [40 , 55] and k uniformly sampled from [20 , 25] .
For minimum dominant set, we follow Dai et al.
[12] to Barabasi-Albert networks [5] with attachment 4 .
Appendix E. More Results E.1. Results on Max Clique, MDS, Min Cut
Here, we report the results for maximum clique, Minimum dominating set, and minimum cut.
The analysis for maximum clique and minimum dominating set are given in the main text.
In minimum cut, we follow Karalias and Loukas [24] and evaluate the performance via local conductance: cut ( S ) / vol ( S ) (the smaller the better).
We can see that the annealed training substantially improves the performance of Erdos across all problem types and all datasets, except for
SF-295 in minimum cut, by utilizing a better-unsupervised training framework.
Our method also outperforms greedy heuristics, classical algorithms such as MFA, CRD, MQI, and other learning based approaches such as RUNCSP, L1/L2 GNN.
[Table] [Table]
E.2. Comparison to Supervised Learning
In order to compare our unsupervised results to supervised results
[Table]
we provide evaluation results for the MDS problem on BA-4 graphs, using the supervised learning result in [12].
As in Table 5, we can see that annealed training significantly improves the performance of unsupervised learning and has a ratio very close to supervised learning.
We also provide a comparison for supervised learning for evaluation on small RB graphs for MIS.
As a source of labels, we use Gurobi to solve the maximum independent set problem.
Due to the computational limitations, we set a time limit as 10 seconds and some of the instances are not solved to optimal.
In this case as in Table 6, we observe that, with the proper training algorithm, the unsupervised learning can even beat the supervised learning
[Table] [Table]
[Table] [Table]
E.3. Evaluation on very large graphs
We conduct extra experiments for MIS on large BA-4 Graphs following Dai et al.
The model is trained on BA-4 graphs with size 1024-1100.
For each larger size, we evaluate the methods on 100 graphs and report the mean, std, and average running time.
We can see that the performance of Gurobi decreases with increasing the graph size and the learning based approaches.
Another observation is that the learning based approaches have much smaller running time, as Dai et al.
[12] has the conditional decoding implemented in cpp, while our conditional decoding is implemented in python.
E.4. Annealing Schedule
We use the schedule (23) so as to make sure the potential change f/
τ k +1 − f/τ k ≡ C is a constant for all steps k .
In fact, with the schedule (23), the potential f/τ k
=
(1 + α ( k − 1))
f/τ 0 is a linear function w.r.t.
k .
Hence, we name it a linear schedule.
It is possible to use other schedules, e.g. f/τ k
= (1 + α ( k − 1))
12 f/τ 0 and f/τ k
= (1 + α ( k − 1))
3 f/τ 0 , and we name them as concave and convex schedule.
The visualization of the temperature schedule and the potential schedule is given in Figure 5.
The initial temperature is also an important hyperparameter.
We evaluate the initial temperature τ 0 = {0.0, 0.1, 0.5, 1.0, 2.0, 5.0}.
We report the results in Figure 5.
We see that the performance is robust for whichever convex, linear, or concave schedule is used.
The more important factor is the initial temperature τ 0 .
The performance is reduced when τ 0 is too small as the energy-based model (3) is not smooth enough, and the performance is robust when τ 0 is large
[Table]
[Table]
500 epochs 0.
temperature convex linear concave     500 epochs     energy convex linear concave 0.
5.0 initial temperature 0.
ratio convex linear concave
Appendix F. Discussion
This paper proposes a generic unsupervised learning framework for combinatorial optimization problems and substantially improves the performance of the state-of-the-art method.
One restriction of the current method is that it relies on condtional decoding to samle solutions from the learned variational distributions.
For problems with more complex constraints, the decoded solutions might be infeasible.
Hence, we believe better decoding strategies should be considered in future work.
The framework’s success relies on smoothing the loss function via critical penalty coefficients and annealed training as they effectively prevent the training from being stuck at local optima.
The techniques introduced here can be potentially applied in a broader context beyond combinatorial optimization, especially in the weakly supervised learning setting like logic reasoning [22], program induction [10], question answering [41] where fine-grained supervisions are missing and required to be inferred.