Title: T2T: From Distribution Learning in Training to Gradient Search in Testing for Combinatorial Optimization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Machine Learning

Combinatorial Optimization

Generative Modeling

Diffusion Model
Abstract: Extensive experiments have gradually revealed the potential performance bottleneck of modeling Combinatorial Optimization (CO) solving as neural solution prediction tasks. The neural networks, in their pursuit of minimizing the average objective score across the distribution of historical problem instances, diverge from the core target of CO of seeking optimal solutions for every test instance. This calls for an effective search on each problem instance, while the model should serve to provide supporting knowledge that benefits the search. To this end, we propose T2T (Training to Testing) framework that first leverages the generative modeling to estimate the high-quality solution distribution for each instance during training, and then conducts a gradient-based search within the solution space during testing. The proposed neural search paradigm consistently leverages generative modeling, specifically diffusion, for graduated solution improvement. It disrupts the local structure of the given solution by introducing noise and reconstructs a lower-cost solution guided by the optimization objective.  Experimental results on Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS) show the significant superiority of T2T, demonstrating an average performance gain of 49.15% for TSP solving and 17.27% for MIS solving compared to the previous state-of-the-art.

T2T: From Distribution Learning in Training to Gradient Search in Testing for Combinatorial Optimization

Abstract

Extensive experiments have gradually revealed the potential performance bottleneck of modeling Combinatorial Optimization (CO) solving as neural solution prediction tasks. The neural networks, in their pursuit of minimizing the average objective score across the distribution of historical problem instances, diverge from the core target of CO of seeking optimal solutions for every test instance. This calls for an effective search on each problem instance, while the model should serve to provide supporting knowledge that benefits the search. To this end, we propose T2T (Training to Testing) framework that first leverages the generative modeling to estimate the high-quality solution distribution for each instance during training, and then conducts a gradient-based search within the solution space during testing. The proposed neural search paradigm consistently leverages generative modeling, specifically diffusion, for graduated solution improvement. It disrupts the local structure of the given solution by introducing noise and reconstructs a lower-cost solution guided by the optimization objective. Experimental results on Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS) show the significant superiority of T2T, demonstrating an average performance gain of 49.15% for TSP solving and 17.27% for MIS solving compared to the previous state-of-the-art.

1 Introduction

Machine Learning (ML) for Combinatorial Optimization (CO) 2 , abbreviated as ML4CO [1; 2; 3; 4], is a rapidly growing area, lying at the intersection between the two well-established communities. CO problems are essential in operational research as well as computer science, in both theory and practice. However, due to the inherent computational difficulty, e.g. NP-hardness, (approximately) solving these problems with effectiveness and efficiency poses significant challenges. Compared with traditional heuristic-based solvers, ML has recently shown its promising potential to automatically learn heuristics in a data-driven scheme [1; 2; 3; 4], or meanwhile with the aid of human knowledge which would normally improve the sample-efficiency [5; 6; 7; 8; 9]. Moreover, the introduction of neural networks also allows for the speedup by parallel matrix computation in forward inference.

A popular framework in ML4CO is developing neural networks to directly predict a solution or generate a solution with the aid of neural predictions, such that the corresponding objective score is minimized [10; 11; 12; 13]. However, the neural network can only fit limited mappings from problem

Distribution

Learning

Gradient

Search

𝑝

!

(𝒔|𝑮)

𝒔

𝑮

Instance

Solution

Distribution

Solution

𝒙

!

.

.

.

𝒙

"

𝒙

!"#

𝑝

$

𝒙

!"#

𝒙

!

, 𝐺)

𝑞 𝒙

!

𝒙

!"#

)

.

.

.

𝒙

#

𝐺

e.g., 2D-Coordinates in TSP

Denoising Process

Noising Process

Solution

parameters to (near) optimal solutions in the training set, resulting in subpar performance on unseen test instances [14; 15]. Therefore, it calls for neural solvers to efficiently explore solutions tailored to each specific problem instance. In the traditional framework, the network prediction typically converges to a local optimum with little diversity, struggling to further explore superior solutions. In this paper, we resort to a more effective framework that first learns to model the distribution of highquality solutions conditioned on problem instances, then conducts a gradient-based search to obtain a superior solution specific to the given instance, as shown in Fig. 1. Existing alternative methods that conduct the tailored search in the testing phase, e.g. active search [16; 17] and meta-learning [18], require updating the model weights during testing, hindering their applications in computational intensive scenarios. In contrast, we develop a more efficient gradient search framework to obtain a superior solution specific to the given instance, without the need to update model weights.

We exploit the powerful generative neural CO solver [19] that learns to model the distribution of highquality solutions conditioned on specific problem instances. The generative models are welcomed in their ability to output high-quality images [20], texts [21] and graphs [22] with great diversity, whereby these advantages are also welcomed in the ML4CO community. Some recent successes in generative learning of CO are demonstrated in classic CO problems e.g. Traveling Salesman Problem (TSP) and Maximal Independent Set (MIS) (via Diffusion [19] and Variational Auto-Encoder [23]), Satisfiability Problem (SAT) and Mixed-Integer Linear Programming (MILP) (via graph generators [24; 25; 26]), also in more realistic applications e.g. routing for chip design (via Diffusion [27] and Generative Adversarial Network [28]). Notably, the very recent adaptation of the flourishing diffusion models in probabilistic generation [19] has resulted in state-of-the-art performance for solving CO problems e.g. TSP. The diffusion modeling for CO solving is illustrated in Fig. 2, where the model is designed to estimate the solution distribution given a certain instance G , i.e. p θ ( x 0 , G ) . Despite its high-quality solutions, a major drawback is the unawareness of optimizing the CO objective as the generative model simply learns to fit a distribution of solutions. To further improve the efficiency of the current costly from-scratch sampling, it stands crucial to introduce a more efficient search-based paradigm to further explore high-quality solutions in terms of the final objective of the problem.

To this end, we propose a gradient-based search paradigm to further leverage the learned solution distribution through training, bridging the solution distribution learning during generative modeling t raining to gradient search during t esting for CO , dubbed T2T. The proposed search paradigm well aligns with the behavior of a progressive generative model especially for the SOTA embodiment i.e. diffusion. The diffusion model provides a probabilistic framework that enables a smooth and flexible transition between random noises and feasible solutions, allowing for varying degrees of structure disruption, with a reference denoising process to restore a solution. Leveraging this framework, it is natural to model the local search heuristic as two stages, i.e., disrupting the local structure and reconstructing it with a better objective score. Specifically, T2T first adds certain noise to the given solution and then denoises it to reconstruct a lower-cost solution in iterations, which is achieved by incorporating the objective optimization guidance into the denoising steps through gradient feedback in the loop, steering the recovered sampling towards the optimization direction, as illustrated in Fig. 3. T2T enhances the solving paradigm by incorporating a search procedure with explicit objective guidance, which enables more efficient sampling and allows for deeper exploitation of the estimated solution distribution.

For the discrete nature of CO, T2T builds itself upon discrete diffusion models [29; 30; 31], which in training learns the transition between categorical noises and feasible discrete solutions that are

represented as probability maps of nodes/edges on graphs indicating whether each node/edge is included in the solution. The transition is tailored for each instance as it is conditioned on the corresponding instance's statistics. The training of T2T follows the standard training paradigm of diffusion models, which primarily equips the model with feasibility awareness and estimates the distribution of high-quality solutions. While during testing, we introduce the objective optimization guidance which serves as the complementary optimization and gradient search utility.

We showcase the efficacy of T2T on two typical NP-hard combinatorial optimization problems for edge-selecting and node-selecting types respectively, i.e., TSP and MIS, which have been attracting most attention in the community of ML4CO. Empirically, T2T exhibits strong performance superiority over existing state-of-the-art neural solvers on benchmark datasets across various scales. Notably, a significant performance gain is observed when compared to the purely constructive diffusion solver [19], thereby highlighting the effectiveness of the designed search-based paradigm.

The highlights of this paper include: 1) We introduce the framework of T2T which first estimates the high-quality solution distribution for each instance during training, and then leverages the diffusion process to perform an effective local search in the solution space during testing; 2) We incorporate the optimization objective guidance into the denoising model with theoretical analysis, which equips the model with gradient feedback directly from the instance-wise objective score, enabling it to perform optimization while ensuring general feasibility; 3) Extensive experiments show the promising performance of the proposed T2T, even when trained without high-quality solution labels.

2 Related Work

Machine Learning for CO Solving. MLsolvers can be categorized into construction solvers and improvement solvers. Constructive solvers include autoregressive solvers [32; 12; 6; 23; 8] which step-by-step extend a partial solution to produce a complete feasible solution, and non-autoregressive solvers [33; 34; 14; 35; 19] which predict a solution (typically soft constrained) in one shot. Improvement solvers [36; 37; 5] often define local operators such as 2OPT [36; 37], node swapping [5; 37], sub-problem resolving [7; 38] to perform local search towards minimizing the optimization objective. The algorithm pipelines generally involve selecting specific regions and locally optimizing sub-solutions. While T2T can be a rather different improvement solver, which allows for parallel optimization across multiple sub-regions and equips the generative modeling with the search procedure.

Generative modeling for CO leverages the powerful expressive and modeling capabilities of generative models for direct CO solving (e.g. V AE [23; 39], GAN [28; 40; 41], Diffusion [19; 27; 42], as well as GFlowNets [43; 44], which, while not a typical generative model itself, is closely related to generative modeling), and for supporting CO solving [24; 25]. DIFUSCO [19] has become state-of-the-art for the TSP problem by leveraging powerful diffusion models. However, no instance-specific search paradigms are introduced in existing methods to fully utilize the estimated solution distribution. Instead, they rely on multiple from-scratch samplings to leverage the distribution, which is computationally expensive. Though [23] attempts to utilize the evolution algorithm to conduct multiple iterations of sampling, it yet incurs even more significant costs without explicit objective guidance in the loop. In contrast, T2T serves as an improvement solver that locally modifies the given solution with objective guidance in the loop. It offers the flexibility to introduce any degree of disruption and rewrite without the necessity to construct the solution from scratch.

Diffusion Probabilistic Models. Diffusion models (score-based models) typically comprise a noising process and a learnable denoising process, where neural networks predict the noise introduced in each step. Diffusion's inference can be seen as a score (gradient) guided search, which is a general principle across different settings [45; 46; 47]. Diffusion in continuous space (e.g., for image generation) [29; 48; 42; 49; 50; 51; 52] have established and consolidated the theoretical foundations of the general diffusion framework. Several landmark works have advanced the diffusion model through various aspects. DDIM [49] generalizes DDPM via a class of non-Markovian processes which gives rise to implicit models that produce high-quality samples much faster. For conditional generation, classifier guidance [29; 52] and classifier-free guidance [53] are introduced with the score estimate of the diffusion models to produce conditional results. Similar paradigms are adopted in discrete diffusion models for generating discrete data using binomial noises [29] or multinomial/categorial noises [30; 31]. Diffusion also demonstrates its capability to learn offline policies in reinforcement learning tasks [54; 55].

3 Approach

This section presents our proposed T2T framework, including the preliminaries, offline training pipeline based on generative modeling, and online testing pipeline with objective guided search.

3.1 Preliminaries: Combinatorial Optimization on Graphs

Following the notations adopted in [13; 56; 57] we define G as the universe of CO problem instances represented by graphs G ( V, E ) ∈ G , where V and E denote the node set and edge set respectively. CO problems can be broadly classified into two types based on the solution composition: edgeselecting problems which involve selecting a subset of edges, and node-selecting problems which select a subset of nodes. Let x ∈ { 0 , 1 } N denote the optimization variable. For edge-selecting problems, N = n 2 and x i · n + j indicates whether E ij is included in x . For node-selecting problems, N = n and x i indicates whether V i is included in x . The feasible set Ω consists of x satisfying specific constraints as feasible solutions. A CO problem on G aims to find a feasible x that minimize the given objective function f ( · ; G ) : { 0 , 1 } N → R ≥ 0 :

In this paper, we study two primary and representative CO problems: TSP and MIS. TSP defines on an undirected complete graph G = ( V, E ) , where V represents n cities and each edge E ij has a non-negative weight w ij representing the distance between cities i and j . The problem can then be formulated as finding a Hamiltonian cycle of minimum weight in G . For MIS, given an undirected graph G = ( V, E ) , an independent set is a subset of vertices S ⊆ V such that no two vertices in S are adjacent in G . MIS is the problem of finding an independent set of maximum cardinality in G .

3.2 Offline Solution Distribution Learning on Training Set based on Diffusion

The basic diffusion modeling for solution generation in training is primarily based on [19]'s design while this paper explicitly formulate the solving task as a conditional generation task. As mentioned in Sec. 3.1, the solutions of CO problems can be represented as x ∈ { 0 , 1 } N with x ∈ Ω . For each entry, the model estimates a Bernoulli distribution indicating whether this entry should be selected. In implementation, each entry of solution x is represented by a one-hot vector 3 such that x ∈ { 0 , 1 } N × 2 . Generative modeling aims to model the distribution of high-quality solutions conditioned on the given instance, while feasibility constraints can be broadly captured through learning and eventually hard-guaranteed by post-processing. For the purpose of problem-solving, the generative model is desired to estimate the solution distribution given a certain instance G , i.e., p θ ( x | G ) . In this section, we model this distribution through the discrete diffusion models [42; 30].

Following the notations of [42; 30], the general framework of diffusion includes a forward noising and a reverse denoising Markov process. The noising process, also denoted as the diffusion process, takes the initial solution x 0 sampled from the distribution q ( x 0 | G ) and progressively introduces noise to generate a sequence of latent variables x 1: T = x 1 , x 2 , · · · , x T . Specifically, the noising process is formulated as q ( x 1: T | x 0 ) = ∏ T t =1 q ( x t | x t -1 ) . The denoising process is learned by the model, which starts from the final latent variable x T and denoises x t at each time step to generate the preceding variables x t -1 based on the instance G , eventually recovering the target data distribution. The formulation of the denoising process is expressed as p θ ( x 0: T | G ) = p ( x T ) ∏ T t =1 p θ ( x t -1 | x t , G ) , where θ denotes the parameters of the model. The training optimization aims to align p θ ( x 0 | G ) with the data distribution q ( x 0 | G ) and the objective adopts the variational upper bound of the negative log-likelihood:

Specifically, the forward noising process is achieved by multiplying x t ∈ [0 , 1] N × 2 at step t with a forward transition probability matrix Q t ∈ [0 , 1] 2 × 2 where [ Q t ] ij indicates the probability of transforming e i in each entry of x t to e j . We set Q t = [ β t 1 -β t 1 -β t β t ] [30], where β t ∈ [0 , 1]

such that the transition matrix is doubly stochastic with strictly positive entries, ensuring that the stationary distribution is uniform which is an unbiased prior for sampling. The noising process for each step and the t -step marginal are formulated as:

where Cat ( x ; p ) is a categorical distribution over N one-hot variables with probabilities given by vector p and Q t = Q 1 Q 2 · · · Q t . Through Bayes' theorem, the posterior can be achieved as:

The neural network is trained to predict the logits of the distribution ˜ p θ (˜ x 0 | x t , G ) , such that the denoising process can be parameterized through q ( x t -1 | x t , ˜ x 0 ) :

Specifically for implementation, the condition G is enforced as an input to the network θ , which is embodied as an anisotropic graph neural network with edge gating mechanisms [33] following [19]. For TSP, the instance condition consists of 2D coordinates of the vertices. The network input includes node embeddings from the 2D coordinates, along with edge embeddings from x t . For MIS, the edges E serve as the instance condition, complemented by the inclusion of node embeddings from x t to collectively form the input. The output of the network is ˜ p θ (˜ x 0 | x t , G ) ∈ [0 , 1] N × 2 parameterizing N Bernoulli distributions for N entries in ˜ x 0 .

3.3 Test-stage Gradient-based Search with Instance-wise Objective Feedback

In this section, we first present the technical solution for the objective gradient incorporation in denoising steps, and then illustrate the neural search paradigm that leverages the guided denoising steps.

3.3.1 Objective-aware Denoising Process with Gradient Feedback

In the context of combinatorial optimization, the incorporation of objective optimization is necessary and important, which enables the direct involvement of objective and effective search over the solution space towards minimizing the score. As discussed in Sec. 3.2, at step t the model denoise x t by p θ ( x t -1 | x t , G ) . For the purpose of objective optimization, we aim to estimate p θ ( x t -1 | x t , G, y ∗ ) where y ∗ is the optimal objective score given the instance G :

to guide the denoising process to x ∗ = arg min x l ( x ; G ) . We will show later that this ability does not necessarily require the access of y ∗ or x ∗ , and can readily be achieved by the model learned in Sec. 3.2 without relying on any additionally trained networks. The adopted objective is defined below.

Optimization Objective. Since the generated samples are not guaranteed to satisfy the hard constraints, we incorporate constraint satisfaction into the objective and utilize a relaxed objective with constraint penalty, as adopted in [13; 56; 57]. Consider a relaxed cost function f r ( · ; G ) : { 0 , 1 } N → R ≥ 0 which defined on the constraint relaxed space { 0 , 1 } N satisfying f r ( x ; G ) = f ( x ; G ) , ∀ x ∈ Ω , and a constraint penalty g r ( x ; G ) : { 0 , 1 } N → R ≥ 0 where g r ( x ; G ) = 0 if x ∈ Ω and g r ( x ; G ) ≥ 1 if x / ∈ Ω , the objective can be formulated as:

where β > max x ∈ Ω f ( x ; G ) . For MIS, the constraint can be straightforwardly modeled as x i x j = 0 if ( i, j ) ∈ E , thus the objective can be formulated as l MIS ( x ; G ) ≜ -∑ 1 ≤ i ≤ N x i + β ∑ ( i,j ) ∈ E x i x j . While for TSP, since the constraints are difficult to explicitly express, we directly select the objective as l TSP = x ⊙ D where D ∈ R n × n + denotes the distance matrix.

Objective Guidance in Denoising Steps. The estimation of p θ ( x t -1 | x t , G, y ∗ ) can be modeled through the following proposition, adapted from the classifier guidance technique [52]:

Proposition 1. The optimization-enforced denoising probability estimation p θ ( x t -1 | x t , G, y ∗ ) equals to Zp θ ( x t -1 | x t , G ) p ( y ∗ | x t -1 , G ) , where Z is a normalizing constant.

The proof is given in the Appendix. As p θ ( x t -1 | x t , G ) can be readily obtained from the trained network, the main challenge of estimating p θ ( x t -1 | x t , G, y ∗ ) lies in estimating p ( y ∗ | x t -1 , G ) . Since x t -1 is not accessible at step t , we utilize Taylor expansion to approximate log p ( y ∗ | x t -1 , G ) around x t -1 = x t , given that x t -1 ∼ x t :

where C ( x t ) is irrelevant to x t -1 . Applying exponentiation, we obtain:

To determine p ( y ∗ | x t , G ) , we utilize energy-based modeling [58] to model the distribution p ( y | x t , G ) . In this context, we define the energy function as:

where x 0 ( x t ) denotes the clean target sample x 0 corresponding to x t . The energy function quantifies the compatibility between y and ( x t , G ) , and it reaches zero when y is exactly the final objective score originating from x t with respect to G . Such a design enables the best y matching the inputs to maintain the highest probability density, and the probability density is positively correlated with the matching degree. Then we employ the Gibbs distribution to characterize the probability distribution over a collection of arbitrary energies:

Let Z = ∫ y ′ exp( -| y ′ -l ( x 0 ( x t ); G ) | ) and substitute in y ∗ , we have:

Note the gradient operator has removed to effect of y ∗ as a constant, thereby eliminating the need to access y ∗ for achieving the target. And Z as the integral over the entire distribution is not affected by x t and thus can also be removed by the gradient operator. Recall that the network θ is trained to predict the logits of ˜ p θ (˜ x 0 | x t ) ∈ [0 , 1] N × 2 which includes the logits of N Bernoulli distributions, satisfying [˜ p θ (˜ x 0 | x t )] i = ( 1 -[ E ˜ x 0 ] i , [ E ˜ x 0 ] i ) . Thus l ( x 0 ( x t ); G ) can be estimated using E ˜ x 0 as:

By Eq. 9, 12 and 13, we obtain p ( y ∗ | x t -1 , G ) ∝ exp ( [ -∇ x t l ( E ˜ x 0 ∼ ˜ p θ (˜ x 0 | x t ) ˜ x 0 ; G ) ] ⊤ x t -1 ) , and the guided denoising probability estimation can now be achieved by p θ ( x t -1 | x t , G, y ∗ ) = Zp θ ( x t -1 | x t , G ) p ( y ∗ | x t -1 , G ) where Z is the normalizing constant, to realize the guided denoising.

3.3.2 The T2T Framework for Efficient Differentiable Neural Search

With the objective gradient incorporated denoising process in Sec. 3.3.1, we now delve into T2T.

Overview. As shown in Fig. 3, the algorithm starts with an initial solution s 0 and conducts several iterations to enhance the given solution. Each iteration involves adding a certain degree of noise to disrupt the structure, denoising with objective gradient guidance to obtain a lower-cost softconstrained solution, and subsequently post-processing to decode a feasible solution s ∈ Ω . The algorithm eventually reports the solution with the lowest objective score ever achieved.

Structure Disruption. Recall that x 1 , x 2 , · · · , x T are situated in spaces characterized by progressively escalating levels of disruption. To introduce a controlled degree of disruption to the given solution, we employ s 0 Q αT to derive the distribution of the disrupted solution q ( s αT | s 0 ) as N Bernoulli distributions, where α serves as a hyperparameter to control the degree of noise. Subsequently, the disrupted solution s αT can be sampled from q ( s αT | s 0 ) .

Objective Guided Solution Reconstruction. From s αT , we employ p θ ( s t -1 | s t , G, y ∗ ) proposed in Sec. 3.3.1 to perform denoising, which aimed at recovering a potentially lower-cost s ′ 0 . Additionally,

Training

Testing

𝑝

!

𝒔

"

#

𝒔

$%

, 𝑦

∗

, 𝐺)

𝑝

!

𝒔

"

##

𝒔

$%

#

, 𝑦

∗

, 𝐺)

Denoising

Denoising

Noising

Noising

Denoising

Noising

2ndRound

ptimization

1stRound

q(sαr I s°)

Solution Distribution

SαT

So

InitialSolution

X

q(sar I so)

XT

xαT

Feasibility

we utilize the fast sampling algorithm introduced in DDIM [49] to accelerate sampling. The algorithm determines a M -element subset from latent variables s 1: αT to obtain { s τ 1 , s τ 2 , · · · , s τ M } ( τ 1 = 1 and τ M = αT ), and directly models p θ ( s τ i -1 | s τ i , G, y ∗ ) , which effectively reduces the number of denoising steps. In our implementation, we uniformly perform a 10-step reconstruction, i.e., M = 10 .

Post-inference Decoding. Note the mechanism guarantees that the recovered solution s ∈ { 0 , 1 } N × 2 (consisting of N one-hot vectors), but it does not necessarily satisfy s ∈ Ω . Thus, in the last step's inference, we directly output the logits of its distribution p θ ( s ′ 0 ) to produce E s ′ 0 as the heatmap H ∈ [0 , 1] N where each element denotes each edge/node's confidence to be selected, and then we adopt greedy decoding to obtain a feasible solution. We follow previous works [59; 35; 19] to perform greedy decoding by sequentially inserting edges (for TSP) or nodes (for MIS) with the highest confidence if there are no conflicts. For TSP, the 2OPT heuristic [60] is optionally applied.

Tradeoff Between Guidance Accuracy and Local Rewriting Degree. Note that the objectiveguided denoising process relies heavily on the accuracy of the estimation ˜ p θ (˜ x 0 | x t ) . Intuitively, it is easier to restore the original solution from the data that has lower noise content. And with excessive noise interference, the estimation will deviate significantly from the ground truth, resulting in faulty guidance. This is also one of the reasons why we use a local rewriting mechanism. However, while the accuracy of the estimation increases as it moves closer to the original solution space, it also limits the degree of reconstruction in terms of the solution structure. Therefore, it is crucial to strike a balance between the estimation accuracy and the extent of local rewriting. To accomplish this, we employ the aforementioned hyperparameter α to control an appropriate range of perturbation.

4 Experiments

All the experiments are performed on GPUs of NVIDIA Telsla A100. All the test evaluations are performed on a single GPU, while the trainings are conducted in parallel with four A100 GPUs. Source code is publicly available at https://github.com/Thinklab-SJTU/T2TCO .

4.1 Experimental Setup

We test on two CO problems including TSP and MIS. The comparison includes SOTA learning-based solvers, heuristics, and exact solvers for each problem. For comparison fairness of learning solvers, the initial solutions of our approach are obtained by denoising random noises by Eq. 5 (same solution as the constructive diffusion solver DIFUSCO [19]). To balance the inference time, for experiments with runtime measurement, T2T utilizes a reduced number of sampling steps for constructing the initial solutions, specifically 20 steps. While DIFUSCO, as an important baseline, adopts 120 inference steps for TSP and 50 steps for MIS. The models for both TSP and MIS are trained with 1000 denoising steps, i.e., T = 1000 . For gradient search, T2T generally involves 3 iterations with 10 guided denoising steps for each. See more details in Appendix.

4.2 Experiments for TSP

Datasets. ATSP instance includes N 2 -D coordinates and a reference solution obtained by heuristics. Training and testing instances are generated via uniformly sampling N nodes from the unit square [0 , 1] 2 , which is a standard procedure as adopted in [12; 23; 33; 61; 35; 19]. We experiment on various problem scales including TSP-50, TSP-100, TSP-500, and TSP-1000. The reference solutions for TSP-50/100 are labeled by the Concorde exact solver [62] and the solutions for TSP-500/1000 are labeled by the LKH-3 heuristic solver [63]. The test set for TSP-50/100 is taken from [12; 33] with 1280 instances and the test set for TSP-500/1000 is from [34] with 128 instances for the fair comparison. This section also incorporates results on real-world TSPLIB 4 dataset.

Metrics. Following [12; 33; 35; 19], we adopt three evaluation metrics: 1) Length: the average total distance or cost of the solved tours w.r.t. the corresponding instances, as directly corresponds to the objective. 2) Drop: the relative performance drop w.r.t. length compared to the global optimality or the reference solution; 3) Time: the average computational time to solve the problems.

Results for TSP-50/100. Table 1 presents the results. Since DIFUSCO [19] has been shown able to attain global optimality for instances with these scales, at the cost of a sampling decoding scheme, we degrade to the greedy decoding setting for a more meaningful evaluation. The baselines include state-of-the-art learning-based methods with greedy decoding, as well as traditional exact and heuristic solvers. Hyperparameter α is set as 0 . 25 . Both DIFUSCO and T2T adopt 50 inference steps. As evident from the results, in the setting of pure greedy decoding without 2OPT refinement, the proposed search-based paradigm achieves a substantial reduction in performance drops from 0.48% to 0.04% for TSP-50 and from 1.01% to 0.18% for TSP-100.

Results for TSP-500/1000. Table 2 presents the results. The baselines include recent learning methods with greedy decoding and sampling decoding ( × 4 ), i.e. producing multiple solutions in parallel and selecting the best one, as well as traditional exact and heuristic solvers. DIFUSCO and T2T are compared in the same conditions. DIFUSCO and T2T adopt 120 and 20 inference steps, respectively. The results of other baselines are quoted from [34] and [35], with the runtimes provided for reference. In the sampling decoding setting, compared to previous neural solvers, T2T improves the performance drop from 0.83% to 0.37% for TSP-500 and 1.30% to 0.78% for TSP-1000.

Effect of Disruption Ratio α . As discussed in Sec. 3.3.2, the hyperparameter α is utilized to regulate the extent of perturbation, striking a balance between guidance accuracy and local rewriting degree. Fig. 4 (a) illustrates the impact of α on the solving performance. The experiments are performed on TSP-500 with both greedy and sampling decoding settings. The results demonstrate the tradeoff existence and indicate that 0 . 3 ≤ α ≤ 0 . 4 can attain relatively better performance.

1.75

1.0

1.75

1.0

1.50

(%)

0.8

1.50

(%)

0.8

Initial Solution

1.25

sno

Initial Solution

0.6

Ours

P

1.25

Initial Solution

Ours

0.6

Ours

Initial Solution

1.00

1.00

0.75

0.4

0.75

0.4

0.1

0.2

0.3

0.4

0.5

0.1

0.2

0.3

0.4

0.5

20

40

60

80

100

20

40

60

80

100

alpha

alpha

Initial Steps

InitialSteps

(a) α -Drop Curve (Greedy and Sampling)

(b) InitStep-Drop Curve (Greedy and Sampling)

Effect of Initial Solution Quality. Fig. 4 (b) illustrates the performance variation when altering the initial solution quality. The quality is controlled by the number of denoising steps for producing the initial solution. The experiments are performed on TSP-500 with both greedy and sampling decoding settings. As shown, T2T exhibits a relatively stable performance across varying initial solution qualities, based on which we generally utilize 20 steps for constructing the initial solutions.

Generalization Results on Synthetic Data and TSPLIB. Based on the problem set {TSP-50, TSP-100, TSP-500, TSP-1000}, we train the model on a specific problem scale and then evaluate it on all problem scales. Table 4 presents the generalization results of T2T and DIFUSCO with greedy decoding. The results show the satisfying cross-domain generalization ability of T2T, e.g., the model trained on TSP-1000 achieves less than a 2% optimality gap on all other problem scales. Moreover, we evaluate our model trained with random 100-node problems on real-world TSPLIB instances with 50-200 nodes. The compared baselines include DIFUSCO and baselines listed in [68]'s Table 3. Table 3 shows T2T's superiority achieving merely

0.133% optimality gap, 58.3% better than the best previous method. Detailed results of Table 3 and the results on large-scale TSPLIB instances with 200-1000 nodes in Appendix A. For each instance, we normalize the coordinates to [0,1] and solve with the solving algorithms.

Validation of Gradient Search with Worse Distribution Learning in Training. We investigate the performance over 1 15 amount of training data and the model trained with lower-quality feasible solutions produced by farthest insertion heuristic (with 7.50% gap to optimality). Results in Fig. 5 validate the effectiveness of gradient search even if the distribution is poorly learned, suggesting that T2T

stands less sensitive to the quality of labels in training than other peer methods. It further demonstrates the generality of T2T especially when high-quality solution labels are not easily available.

4.3 Experiments for MIS

Datasets. Two datasets are tested for the MIS problem following [70; 72; 71; 35; 19], include SATLIB [73] and Erd˝ os-Rényi (ER) graphs [74]. SATLIB contains SAT instances encoded in DIMACS CNF format, which are then reduced to MIS instances. ER graphs are randomly generated with each edge maintaining a fixed probability of being present or absent, independently of the other edges. We adopt ER graphs of 700 to 800 nodes with the pairwise connection probability set as 0 . 15 .

Metrics. Following previous works [12; 33; 35; 19], we adopt three evaluation metrics to measure model performance: 1) Size: the average size of the solutions w.r.t. the corresponding instances, i.e. the objective. 2) Drop: the relative performance drop w.r.t. length compared to the optimal solution or the reference solution; 3) Time: the average computational time required to solve the problems.

Main Results. Table 5 presents results. The baselines include state-of-the-art neural methods with greedy decoding and sampling decoding ( × 4 ), as well as exact solver Gurobi [66] and heuristic solver KaMIS [69]. The solving time of Gurobi is set as comparable to neural solvers, thus it does not reach optimality. DIFUSCO and T2T are compared in the same conditions. The results of other baselines are quoted from [35]. DIFUSCO and T2T adopt 50 and 20 inference steps, respectively. In the greedy decoding setting, compared to previous neural solvers, T2T improves the performance drop from 0.33% to 0.22% for SATLIB dataset and from 18.53% to 11.83% for ER-[700-800] dataset.

5 Conclusion and Future work

We have presented a principled learning paradigm for solving hard-constrained optimization problems specifically combinatorial optimization, which bridges the offline training with reference solution as supervision to the instance-wise solving in testing which calls for more direct gradient feedback from the objective score. Experiments on TSP and MIS show the strong performance of our approach. With the potential generality, further validation across a wider range of CO problems is the immediate further work, including those with general representing capabilities such as Mixed Integer Programming (MIP) and ML-dominant special problems like Quadratic Assignment Problem (QAP). Another promising future direction is to explore the potential of recent (scalable) graph Transformers [75; 76; 77] as expressive encoder backbones for solving complex CO problems.

References

Y. Bengio, A. Lodi, and A. Prouvost, 'Machine learning for combinatorial optimization: a methodological tour d'horizon,' European Journal of Operational Research , 2021.

Q. Cappart, D. Chételat, E. Khalil, A. Lodi, C. Morris, and P. Veliˇ ckovi´ c, 'Combinatorial optimization and reasoning with graph neural networks,' arXiv preprint arXiv:2102.09544 , 2021.

J. Zhang, C. Liu, X. Li, H.-L. Zhen, M. Yuan, Y. Li, and J. Yan, 'A survey for solving mixed integer programming via machine learning,' Neurocomputing , vol. 519, pp. 205-217, 2023.

W. Guo, H.-L. Zhen, X. Li, W. Luo, M. Yuan, Y. Jin, and J. Yan, 'Machine learning methods in solving the boolean satisfiability problem,' Machine Intelligence Research , 2023.

X. Chen and Y. Tian, 'Learning to perform local rewriting for combinatorial optimization,' Advances in Neural Information Processing Systems , vol. 32, 2019.

Y.-D. Kwon, J. Choo, B. Kim, I. Yoon, Y. Gwon, and S. Min, 'Pomo: Policy optimization with multiple optima for reinforcement learning,' Advances in Neural Information Processing Systems , vol. 33, pp. 21 188-21 198, 2020.

S. Li, Z. Yan, and C. Wu, 'Learning to delegate for large-scale vehicle routing,' Advances in Neural Information Processing Systems , vol. 34, pp. 26 198-26 211, 2021.

M. Kim, J. Park, and J. Park, 'Sym-nco: Leveraging symmetricity for neural combinatorial optimization,' arXiv preprint arXiv:2205.13209 , 2022.

Y. Li, Y. Jia, W. Luo, H.-L. Zhen, X. Li, M. Yuan, and J. Yan, 'Kissat adaptive restart, kissat cfexp: Adaptive restart policy and variable scoring improvement,' Proceedings of SAT Competition , 2022.

O. Vinyals, M. Fortunato, and N. Jaitly, 'Pointer networks,' Advances in neural information processing systems , vol. 28, 2015.

R. Wang, J. Yan, and X. Yang, 'Neural graph matching network: Learning lawler's quadratic assignment problem with extension to hypergraph and multiple-graph matching,' IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

W. Kool, H. Van Hoof, and M. Welling, 'Attention, learn to solve routing problems!' arXiv preprint arXiv:1803.08475 , 2018.

N. Karalias and A. Loukas, 'Erdos goes neural: an unsupervised learning framework for combinatorial optimization on graphs,' Advances in Neural Information Processing Systems , vol. 33, pp. 6659-6672, 2020.

S. Geisler, J. Sommer, J. Schuchardt, A. Bojchevski, and S. Günnemann, 'Generalization of neural combinatorial solvers through the lens of adversarial robustness,' in International Conference on Learning Representations , 2022.

H. Lu, Z. Li, R. Wang, Q. Ren, X. Li, M. Yuan, J. Zeng, X. Yang, and J. Yan, 'Roco: A general framework for evaluating robustness of combinatorial optimization solvers on graphs,' in The Eleventh International Conference on Learning Representations , 2023.

I. Bello, H. Pham, Q. V. Le, M. Norouzi, and S. Bengio, 'Neural combinatorial optimization with reinforcement learning,' arXiv preprint arXiv:1611.09940 , 2016.

A. Hottung, Y.-D. Kwon, and K. Tierney, 'Efficient active search for combinatorial optimization problems,' arXiv preprint arXiv:2106.05126 , 2021.

H. Wang and P. Li, 'Unsupervised learning for combinatorial optimization needs meta-learning,' arXiv preprint arXiv:2301.03116 , 2023.

Z. Sun and Y. Yang, 'Difusco: Graph-based diffusion solvers for combinatorial optimization,' arXiv preprint arXiv:2302.08224 , 2023.

G. Harshvardhan, M. K. Gourisaria, M. Pandey, and S. S. Rautaray, 'A comprehensive survey and analysis of generative models in machine learning,' Computer Science Review , vol. 38, p. 100285, 2020.

T. Iqbal and S. Qureshi, 'The survey: Text generation models in deep learning,' Journal of King Saud University-Computer and Information Sciences , vol. 34, no. 6, pp. 2515-2528, 2022.

X. Guo and L. Zhao, 'A systematic survey on deep generative models for graph generation,' IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022.

A. Hottung, B. Bhandari, and K. Tierney, 'Learning a latent search space for routing problems using variational autoencoders,' in International Conference on Learning Representations , 2021.

Y. Li, X. Chen, W. Guo, X. Li, W. Luo, J. Huang, H.-L. Zhen, M. Yuan, and J. Yan, 'Hardsatgen: Understanding the difficulty of hard sat formula generation and a strong structure-hardnessaware baseline,' in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) , 2023.

J. You, H. Wu, C. Barrett, R. Ramanujan, and J. Leskovec, 'G2sat: Learning to generate sat formulas,' in Advances in neural information processing systems , vol. 32, 2019.

Z. Geng, X. Li, J. Wang, X. Li, Y. Zhang, and F. Wu, 'A deep instance generative framework for milp solvers under limited data availability,' arXiv preprint arXiv:2310.02807 , 2023.

X. Du, C. Wang, R. Zhong, and J. Yan, 'Hubrouter: Learning global routing via hub generation and pin-hub connection,' in Advances in Neural Information Processing Systems , 2023.

R. Cheng, X. Lyu, Y. Li, J. Ye, J. Hao, and J. Yan, 'The policy-gradient placement and generative routing neural networks for chip design,' Advances in Neural Information Processing Systems , vol. 35, pp. 26 350-26 362, 2022.

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, 'Deep unsupervised learning using nonequilibrium thermodynamics,' in International Conference on Machine Learning , 2015, pp. 2256-2265.

J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. van den Berg, 'Structured denoising diffusion models in discrete state-spaces,' Advances in Neural Information Processing Systems , vol. 34, pp. 17 981-17 993, 2021.

E. Hoogeboom, D. Nielsen, P. Jaini, P. Forré, and M. Welling, 'Argmax flows and multinomial diffusion: Learning categorical distributions,' Advances in Neural Information Processing Systems , vol. 34, pp. 12 454-12 465, 2021.

E. Khalil, H. Dai, Y. Zhang, B. Dilkina, and L. Song, 'Learning combinatorial optimization algorithms over graphs,' Advances in neural information processing systems , vol. 30, 2017.

C. K. Joshi, T. Laurent, and X. Bresson, 'An efficient graph convolutional network technique for the travelling salesman problem,' arXiv preprint arXiv:1906.01227 , 2019.

Z.-H. Fu, K.-B. Qiu, and H. Zha, 'Generalize a small pre-trained model to arbitrarily large tsp instances,' in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 8, 2021, pp. 7474-7482.

R. Qiu, Z. Sun, and Y. Yang, 'Dimes: A differentiable meta solver for combinatorial optimization problems,' arXiv preprint arXiv:2210.04123 , 2022.

P. R. d O Costa, J. Rhuggenaath, Y. Zhang, and A. Akcay, 'Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning,' in Asian Conference on Machine Learning , 2020, pp. 465-480.

Y. Wu, W. Song, Z. Cao, J. Zhang, and A. Lim, 'Learning improvement heuristics for solving routing problems,' IEEE transactions on neural networks and learning systems , vol. 33, no. 9, pp. 5057-5069, 2021.

Q. Hou, J. Yang, Y. Su, X. Wang, and Y. Deng, 'Generalize learned heuristics to solve largescale vehicle routing problems in real-time,' in The Eleventh International Conference on Learning Representations , 2023.

D. P. Kingma and M. Welling, 'Auto-encoding variational bayes,' in ICLR , 2014.

Y. Li, Y. Mo, L. Shi, and J. Yan, 'Improving generative adversarial networks via adversarial learning in latent space,' in Advances in Neural Information Processing Systems , 2022.

Y. Li, L. Shi, and J. Yan, 'Iid-gan: an iid sampling perspective for regularizing mode collapse,' in International Joint Conference on Artificial Intelligence , 2023.

J. Ho, A. Jain, and P. Abbeel, 'Denoising diffusion probabilistic models,' Advances in Neural Information Processing Systems , vol. 33, pp. 6840-6851, 2020.

D. Zhang, H. Dai, N. Malkin, A. Courville, Y. Bengio, and L. Pan, 'Let the flows tell: Solving graph combinatorial optimization problems with gflownets,' arXiv preprint arXiv:2305.17010 , 2023.

Y. Bengio, S. Lahlou, T. Deleu, E. J. Hu, M. Tiwari, and E. Bengio, 'Gflownet foundations,' arXiv preprint arXiv:2111.09266 , 2021.

D. Belanger, B. Yang, and A. McCallum, 'End-to-end learning for structured prediction energy networks,' in International Conference on Machine Learning . PMLR, 2017, pp. 429-439.

Y. Chen, Y. Shi, and B. Zhang, 'Optimal control via neural networks: A convex approach,' arXiv preprint arXiv:1805.11835 , 2018.

B. Amos, L. Xu, and J. Z. Kolter, 'Input convex neural networks,' in International Conference on Machine Learning . PMLR, 2017, pp. 146-155.

Y. Song and S. Ermon, 'Generative modeling by estimating gradients of the data distribution,' Advances in neural information processing systems , vol. 32, 2019.

J. Song, C. Meng, and S. Ermon, 'Denoising diffusion implicit models,' arXiv preprint arXiv:2010.02502 , 2020.

Y. Song and S. Ermon, 'Improved techniques for training score-based generative models,' Advances in neural information processing systems , vol. 33, pp. 12 438-12 448, 2020.

A. Q. Nichol and P. Dhariwal, 'Improved denoising diffusion probabilistic models,' in International Conference on Machine Learning , 2021, pp. 8162-8171.

P. Dhariwal and A. Nichol, 'Diffusion models beat gans on image synthesis,' Advances in Neural Information Processing Systems , vol. 34, pp. 8780-8794, 2021.

J. Ho and T. Salimans, 'Classifier-free diffusion guidance,' arXiv preprint arXiv:2207.12598 , 2022.

Z. Wang, J. J. Hunt, and M. Zhou, 'Diffusion policies as an expressive policy class for offline reinforcement learning,' in The Eleventh International Conference on Learning Representations , 2022.

H. He, C. Bai, K. Xu, Z. Yang, W. Zhang, D. Wang, B. Zhao, and X. Li, 'Diffusion model is an effective planner and data synthesizer for multi-task reinforcement learning,' in Advances in Neural Information Processing Systems , 2023.

H. P. Wang, N. Wu, H. Yang, C. Hao, and P. Li, 'Unsupervised learning for combinatorial optimization with principled objective relaxation,' in Advances in Neural Information Processing Systems , 2022.

M. J. Schuetz, J. K. Brubaker, and H. G. Katzgraber, 'Combinatorial optimization with physicsinspired graph neural networks,' Nature Machine Intelligence , vol. 4, no. 4, pp. 367-377, 2022.

Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, and F. Huang, 'A tutorial on energy-based learning,' Predicting structured data , vol. 1, no. 0, 2006.

A. Graikos, N. Malkin, N. Jojic, and D. Samaras, 'Diffusion models as plug-and-play priors,' arXiv preprint arXiv:2206.09012 , 2022.

S. Lin and B. W. Kernighan, 'An effective heuristic algorithm for the traveling-salesman problem,' Operations research , vol. 21, no. 2, pp. 498-516, 1973.

P. R. d. O. da Costa, J. Rhuggenaath, Y. Zhang, and A. Akcay, 'Learning 2-opt heuristics for the traveling salesman problem via deep reinforcement learning,' arXiv preprint arXiv:2004.01608 , 2020.

D. Applegate, R. Bixby, V. Chvatal, and W. Cook, 'Concorde tsp solver,' 2006.

K. Helsgaun, 'An extension of the lin-kernighan-helsgaun tsp solver for constrained traveling salesman and vehicle routing problems,' Roskilde: Roskilde University , pp. 24-50, 2017.

G. A. Croes, 'A method for solving traveling-salesman problems,' Operations research , vol. 6, no. 6, pp. 791-812, 1958.

X. Bresson and T. Laurent, 'The transformer network for the traveling salesman problem,' arXiv preprint arXiv:2103.03012 , 2021.

Gurobi Optimization, 'Gurobi optimizer reference manual,' http://www.gurobi.com, 2020.

M. Deudon, P. Cournut, A. Lacoste, Y. Adulyasak, and L.-M. Rousseau, 'Learning heuristics for the tsp by policy gradient,' in International conference on the integration of constraint programming, artificial intelligence, and operations research . Springer, 2018, pp. 170-181.

B. Hudson, Q. Li, M. Malencia, and A. Prorok, 'Graph neural network guided local search for the traveling salesperson problem,' arXiv preprint arXiv:2110.05291 , 2021.

S. Lamm, P. Sanders, C. Schulz, D. Strash, and R. F. Werneck, 'Finding near-optimal independent sets at scale,' in 2016 Proceedings of the Eighteenth Workshop on Algorithm Engineering and Experiments (ALENEX) . SIAM, 2016, pp. 138-150.

Z. Li, Q. Chen, and V. Koltun, 'Combinatorial optimization with graph convolutional networks and guided tree search,' Advances in neural information processing systems , vol. 31, 2018.

M. Böther, O. Kißig, M. Taraz, S. Cohen, K. Seidel, and T. Friedrich, 'What's wrong with deep learning in tree search for combinatorial optimization,' arXiv preprint arXiv:2201.10494 , 2022.

S. Ahn, Y. Seo, and J. Shin, 'Learning what to defer for maximum independent sets,' in International Conference on Machine Learning , 2020, pp. 134-144.

H. H. Hoos and T. Stützle, 'Satlib: An online resource for research on sat,' Sat , vol. 2000, pp. 283-292, 2000.

P. Erd˝ os, A. Rényi et al. , 'On the evolution of random graphs,' Publ. Math. Inst. Hung. Acad. Sci , vol. 5, no. 1, pp. 17-60, 1960.

Q. Wu, W. Zhao, Z. Li, D. Wipf, and J. Yan, 'Nodeformer: A scalable graph structure learning transformer for node classification,' in Advances in Neural Information Processing Systems , 2022.

Q. Wu, C. Yang, W. Zhao, Y. He, D. Wipf, and J. Yan, 'DIFFormer: Scalable (graph) transformers induced by energy constrained diffusion,' in International Conference on Learning Representations , 2023.

Q. Wu, W. Zhao, C. Yang, H. Zhang, F. Nie, H. Jiang, Y. Bian, and J. Yan, 'Simplifying and empowering transformers for large-graph representations,' in Advances in Neural Information Processing Systems (NeurIPS) , 2023.

J. Zhou, Y. Wu, W. Song, Z. Cao, and J. Zhang, 'Towards omni-generalizable neural methods for vehicle routing problems,' arXiv preprint arXiv:2305.19587 , 2023.

Appendix

A Supplementary Experiments

Results on TSPLIB. We evaluate our model trained with random 100-node problems on real-world TSPLIB instances with 50-200 nodes. The compared baselines include DIFUSCO and baselines listed in [68]'s Table 3. Table 6 presents the results and shows T2T's superiority achieving merely 0.133% optimality gap, 58.3% better than the best previous method. We also supplement the results (optimality drop) of DIFUSCO and T2T on large-scale TSPLIB benchmark instances with 200-1000 nodes, as shown in Table 7. For each instance, we normalize the coordinates to [0,1] and solve with the solving algorithms.

Generalization Results on Distribution Shift. The generalization results of the diffusion-based methods (e.g., DIFUSCO and T2T) on the distribution (e.g., cluster, gaussian, etc., instead of uniform) are presented in Table. 8 and Table. 9. The TSP data of different distributions are from [78].

Effect of Number of Iterations. Table. 10 shows the results of T2T with varying #iterations. Each iteration contributes to a single process of solution improvement, and employing more iterations can lead to improved solutions. However, too many iterations could lead to additional computational overhead. As a result, we aim to strike a balance in this trade-off and set #iterations to 3.

Effect of Graph Sparsification Hyperparameter k . Table. 10 shows the results of T2T with varying k . A larger value of k can yield improved results, yet it can also lead to increased computational overhead. Thus, we strike a balance in this trade-off and set k to 50.

B Applicability of T2T

We provide a summary concerning the types of problems that T2T can effectively manage:

Mandatory Requirements:

-The decision variables are limited to a countable and finite set of values. Currently, we focus on binary decision variables and the model predicts a heatmap for the solution. But it can be easily extended to prediction through the multinomial diffusion. Note that even binary variables can cover a wide range of CO problems especially on graphs, since most problems can be attributed to edge-selecting problems like TSP, CVRP, and node-selecting problems like MIS, MAX Clique.

-A problem-specific post-processing procedure is required to transform the prediction into feasible solutions. For problems featuring binary decision variables, this prerequisite can be easily met with the greedy decoding strategy, which sequentially inserts variables (edges or nodes) with the highest confidence if there are no conflicts.

Nice-to-have Requirements:

-The constraints can be transformed to some sort of penalty with meaningful gradient. While it is preferable to include a constraint penalty term in Eq. 7, the utilization of the optimization objective alone is also effective (as evidenced by the TSP results). This is due to the diffusion's capacity to implicitly learn constraints.

C Proofs

Proposition 1. The optimization-enforced denoising probability estimation p θ ( x t -1 | x t , G, y ∗ ) equals to Zp θ ( x t -1 | x t , G ) p ( y ∗ | x t -1 , G ) , where Z is a normalizing constant.

Proof. Since p θ ( x t -1 | x t , G, y ∗ ) estimates q ( x t -1 | x t , G, y ∗ ) , we aim to show q ( x t -1 | x t , G, y ∗ ) ∝ q ( x t -1 | x t , G ) p ( y ∗ | x t -1 , G ) , such that as q ( x t -1 | x t , G ) can be readily estimated by p θ ( x t -1 | x t , G ) , q ( x t -1 | x t , G, y ∗ ) 's neural estimation p θ ( x t -1 | x t , G, y ∗ ) can be achieved by Zp θ ( x t -1 | x t , G ) p ( y ∗ | x t -1 , G ) , where Z is a normalizing constant.

We first show that p ( y ∗ | x t -1 , G ) does not depend on x t :

Then the probability distribution q ( x t -1 | x t , G, y ∗ ) can be derived by:

q ( y ∗ | x t , G ) is not depend on x t -1 , thus it can be regarded as a constant. Now we can obtain q ( x t -1 | x t , G, y ∗ ) ∝ q ( x t -1 | x t , G ) p ( y ∗ | x t -1 , G ) , completing the proof.

D Discussion from Another Aspect: Constructive and Improvement Solvers

Existing CO solvers including both traditional and emerging ML-based methods, generally can be categorized into two groups as summarized in detail in Table 12. Constructive solvers solve the problem from scratch in one shot or multiple steps, and the algorithm ends once a complete solution is obtained. However, effectively enhancing the final solution quality (e.g. the objective score) while ensuring the constraints can be challenging for constructive solvers due to the absence of explicit reference to complete solutions (as the constraints are often obeyed globally across the dimensions of the solution). While in contrast, improvement solvers yield and maintain a series of complete solutions during the improvement procedure to optimize (namely minimize) the objective in a score decreasing trend (with fluctuation). Compared with constructive solvers, improvement (search-based) solvers [36; 37; 5; 37; 7; 38] seek an enhanced solution by iteratively performing local search within its neighborhood. The algorithm pipelines generally involve selecting specific regions and locally optimizing sub-solutions. This paradigm often offers better tractability and a more direct correlation to objective score thanks to the explicit involvement of the objective and accessibility of complete solutions. While improvement-based solvers dominate in traditional heuristics (e.g., LKH [60; 63]), for learning class methods, the constructive solvers remain relatively major in number, given their more direct compatibility with network prediction tasks.

For constructive ML-solvers, the training procedures can be classified into reinforcement learning (RL) and supervised learning (SL). RL methods [12; 6; 8] try to directly optimize the objective

function, but since a complete solution is not obtained until the end, the intermediate process lacks a valid reference. SL methods [33; 35; 19] typically fit the network prediction to the reference solutions. The lack of objective awareness and local search procedures limits their ability to surpass the supervised solvers. The training procedures of current improvement ML-solvers are based on RL.

Generative modeling for CO [23; 19] currently falls into a subclass of constructive solvers. Generative modeling for CO currently falls into a subclass of constructive solvers, which leverages the powerful expressive and modeling capabilities of generative models, e.g., V AE [23] and the diffusion model [19]. However, as mentioned above the model lacks awareness of the optimization objective and does not introduce a search procedure to fully utilize the estimated solution distribution. In contrast to the constructive way for CO problem solving, our T2T serves as an improvement solver the strong compatibility between the diffusion model and the diffuse-and-denoise local search paradigm with objective feedback in the loop. In our approach, generative diffusion model and local search are both used (which is the first time to our best knowledge), in a complementary and synergistic way to achieve state-of-the-art performance in both objective score and solving time. Compared to existing improvement solvers, such a design facilitates parallel optimization across multiple sub-regions of the entire solution, greatly improving efficiency while also eliminating the need for the iterative selection of optimization regions.

E Network Architecture Details

E.1 Input Embedding Layer

Given node vector x ∈ R N × 2 , weighted edge vector e ∈ R E , denoising timestep t ∈ { τ 1 , . . . , τ M } , where N denotes the number of nodes in the graph, and E denotes the number of edges, we compute the sinusoidal features of each input element respectively:

where d is the embedding dimension, T is a large number (usually selected as 10000 ), concat( · ) denotes concatenation.

Next, we compute the input features of the graph convolution layer:

where t 0 ∈ R d t , d t is the time feature embedding dimension. Specifically, for TSP, the embedding input edge vector e is a weighted adjacency matrix, which represents the distance between different nodes, and e 0 is computed as above. For MIS, we initialize e 0 to a zero matrix 0 E × d .

E.2 Graph Convolution Layer

Following [33], the cross-layer convolution operation is formulated as:

where x l i and e l ij denote the node feature vector and edge feature vector at layer l , W 1 , · · · , W 5 ∈ R h × h denote the model weights, η l ij denotes the dense attention map. The convolution operation integrates the edge feature to accommodate the significance of edges in routing problems.

For TSP, we aggregate the timestep feature with the edge convolutional feature and reformulate the update for edge features as follows:

For MIS, we aggregate the timestep feature with the node convolutional feature and reformulate the update for node features as follows:

E.3 Output Layer

The prediction of the edge heatmap in TSP and node heatmap in MIS is as follows:

where L is the number of GCN layers and norm is layer normalization.

E.4 Hyper-parameters

For both TSP and MIS tasks, we construct a 12-layer GCN derived above. We set the node, edge, and timestep embedding dimension d = 256 , 128 for TSP and MIS tasks, respectively.

F Fast Inference for Discrete Diffusion Models

The technique of denoising diffusion implicit models (DDIMs) [49] is introduced for accelerating inference and solution reconstruction. Let x 1: T denotes the latent variables and { τ 1 , . . . , τ M } is one increasing subset of time sequences { 1 , . . . , T } . τ M = T for the inference processes and αT for the reconstruction processes. The diffusion processes can be generated to a class non-Markovian processes with the same training objective, where the inference processes traverse only on a subset of latent variables. The inference processes are reformulated as:

For both TSP and MIS tasks, we adopt the cosine denosing scheduler such that τ i = ⌊ cos ( 1 -π · ci 2 ) · T ⌋ for some constant c .

G Experimental Details

G.1 Training Resource Requirement

Table 13: Details about the training resource requirement of T2T framework. The results are calculated on A100 GPUs.

We provide the details about the offline training resource requirement of T2T framework in Table 13.

The results are calculated on A100 GPUs. For comparison reference: AM [12] requires 128M instances generated on the fly for training TSP-100, and it requires 45.8h on 2 1080Ti GPUs; POMO [6] requires 200M instances generated on the fly for training TSP-100, and it requires 1 week on a single Titan RTX; Sym-NCO [8] based on POMO requires 2 weeks on a single A100,

Sym-NCO [8] based on AM requires 3 days on 4 A100 GPUs.

G.2 Hyperparamters

Weconduct experiments on TSP and MIS benchmarks with our methods and compare the performance with SOTA learning-based solvers, heuristics, and exact solvers. We select hyperparameters to balance the inference time and the solution quality. Specifically, the inference time of our method is approximate with other learning-based frameworks. For each benchmark, we adopt 20 inference steps and 10 objective-guided denoising steps for reconstruction, which involves 3 iterations and selects the optimal solutions. The disruption ratio α associated with each benchmark is listed in Table. 14.

G.3 Baseline Settings

G.3.1 TSP Benchmarks

TSP-50/100. For TSP-50 and TSP-100, we compare our proposed T2T with 10 baselines, including one exact solver Concorde [62], two heuristic solvers 2OPT [64] and Farthest Insertion, and 7 learning-based solvers, i.e., AM [12], GCN [33], Transformer [65], POMO [6], Sym-NCO [8], Image Diffusion [59] and DIFUSCO [19]. The post-processing includes greedy sampling and 2OPT refinement. To ensure a fair comparison with similar runtimes, we set the number of inference steps for DIFUSCO to 120.

TSP-500/1000. For TSP-500 and TSP-1000, we compare our method with 2 exact solvers Concorde [62] and Gurobi [66], 2 heuritics solvers LKH-3 [63] and Farthest Insertion, and 6 learningbased methods including EAN [67], AM [12], GCN [33], POMO+EAS [17], DIMES [35] and DIFUSCO [19]. The learning-based methods can further be categorized into supervised learning (SL) and reinforcement learning (RL). The post-processing includes greedy sampling (Grdy, G), multiple sampling (S), 2OPT refinement (2OPT), beam search (BS), active search (AS), and their combinations. To ensure a fair comparison with similar runtimes, we set the number of inference steps for DIFUSCO to 120.

G.3.2 MIS Benchmarks

We evaluate our method on two benchmarks, namely SATLIB and ER-[700-800]. For both benchmarks, we compare T2T with one exact solver Gurobi [66], one heuristic solver KaMIS [69] and 5 learning-based frameworks Intel [70], DGL [70], LwD [72], DIMES [35] and DIFUSCO [19]. The post-processing includes greedy sampling (Grdy) and tree search (TS). Specifically, we set the inference steps 50 for DIFUSCO on both benchmarks.

G.4 Graph Sparsification

In TSP, the number of edges in the graph increases quadratically with the number of nodes, which poses scalability challenges. To address this, we adopt graph sparsification by limiting each node to connect only with its k nearest neighbors (KNN) based on Euclidean distance. Specifically, we set k = 50 for TSP-500 and TSP-1000.

G.5 Visualization

Fig. 6 visualizes the specific process of T2T with 5 rounds of gradient search on TSP-50.

𝒕 = 𝜶𝑻

𝒕 = 𝟎

Postprocessing

Round 1

Round 2

Round 3

Round 4

Round 5

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel framework, T2TCO, for solving combinatorial optimization problems. Specifically, it trains a generative model (i.e., discrete diffusion solver from [1]) to estimate the distribution of high-quality solutions for each instance, and, during testing, conducts an effective local search (i.e., iterative objective-aware denoising) guided by the gradient feedback within the estimated solution space. The experiments are conducted on TSP and MIS to demonstrate the superiority over previous SOTA methods.\n\nSTRENGTHS:\n* The writing of this paper is good. The motivation of the proposed method is clear. The drawbacks of previous tailored search methods in the test phase (i.e., active search or meta-learning needs to update model parameters), and that of the constructive diffusion solver [1] (i.e., without explicitly considering the optimization of CO objective during denoising) are discussed clearly.\n\n* The proposed method seems to be sound and novel. This paper uses the discrete diffusion model from [1] to model the high-quality solution distribution for each instance. The novelty of the proposed T2TCO could be mainly attributed to the objective-aware denoising process $p_{\\theta}(x_{t-1}|x_t,G,y^*)$ with gradient feedback (see Section 3.3.1), along with its theoretical analyses.\n\n* The empirical results are good, achieving SOTA on TSP and MIS.\n\nWEAKNESSES:\n* The proposed method is only applicable to diffusion-based methods [1], which may limit the scope of this paper. Moreover, it is an incremental work upon DIFUSCO [1].\n* Computational Complexity:\n  * The training cost of the diffusion model is extremely high, requiring **four A100 GPUs**.  It would be an interesting direction on improving the efficiency of the diffusion-based models for COPs.\n  * Could you provide further details about the needed training time (if training on a single GPU) and GPU memory?\n  * How about the training and inference complexity compared to DIFUSCO [1] (e.g., given the same number of sampling steps)?\n* Empirical Evaluation:\n  * How about the performance if using Monte-Carlo Tree Search (MCTS) [2] as the post-inference decoding?\n  * Any training setups for baselines? POMO [3] could achieve 0.03\\% and 0.14\\% on TSP50 and TSP100, respectively, while you report 0.64\\% and 1.07\\% in Table 1. These differences are not negligible. \n  * How about the generalization performance? DIFUSCO [1] has conducted the generalization study in their empirical analyses. Moreover, recent works [4, 5] consider the generalization study of neural (TSP) solvers across both size and distribution. It is interesting to see how your method performs in this setting.\n  * The detailed results on the classical benchmark dataset (e.g., TSPLIB [6]) are needed for both T2TCO and DIFUSCO.\n* Minor:\n  * In line 96, \"autoregressive\" -> \"construction\".\n  * In line 113, \",T2TCO\" -> \", T2TCO\".\n  * In line 170, \"$Cat$\" -> \"Cat\".\n  * In line 220, \" Note\" -> \"Note\"; \"to\" -> \"the\".\n\n[1] DIFUSCO: Graph-based diffusion solvers for combinatorial optimization. In arXiv:2302.08224.\n\n[2] Generalize a small pre-trained model to arbitrarily large tsp instances. In AAAI 2021.\n\n[3] POMO: Policy optimization with multiple optima for reinforcement learning. In NeurIPS 2020.\n\n[4] On the generalization of neural combinatorial optimization heuristics. In ECMLPKDD 2022.\n\n[5] Towards Omni-generalizable Neural Methods for Vehicle Routing Problems. In ICML 2023.\n\n[6] http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95\n\n*Note to AC: I am not an expert in diffusion and hence cannot completely ensure the correctness of the equations in Section 3.3.1 and the proof in the Appendix.*\n\n----\n\n**Overall**, despite the limited scope and expensive training cost, the proposed method is sound and novel, which may bring new insights to the ML4CO or NCO community. The empirical results are good as well. Therefore, I lean towards borderline acceptance.\n\nQUESTIONS:\n* Do you plan to release the source code during the rebuttal period?\n* Has a similar idea (to the one in Section 3.3.1) been proposed in other domains (e.g., computer vision)?\n* Could you discuss further the objective function in Eqs. (6)-(7)? For example, why not use Eq. (1) in [1]?\n* Why the energy function is designed as Eq. (10)?\n* In Eq. (12), why $-\\log\\nabla_{x_t}Z$ is removed? It seems that $Z$ is also a function of $x_t$.\n* In line 224, is the negative sign missing in the exponentiation, i.e., $\\exp(-\\cdots)$?\n* The authors mention that $x_{t-1}$ is not accessible at step $t$. However, the approximation of $p(y^*|x_{t-1},G)$ still depends on $x_{t-1}$ (in line 224). I'm confused about it. Could you explain it further?\n\nLIMITATIONS:\nLimited scope and training efficiency.\n\nNo negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a diffusion model based solution distribution learning method for combinatorial optimization. The main novelty is that a search mechanism is proposed, which uses the learned distribution to conduct a gradient based search for each instance where the gradient is computed w.r.t the optimiztion objective function. Comparing with existing testing-time gradient update methods such as active search and meta-learning, this method does not need to update model parameters, hence could be more efficient. Experiments on two graph problems TSP (up to 1000 nodes) and MIS show that the proposed method outperforms recent neural CO methods.\n\nSTRENGTHS:\n1. The idea of using learned distribution to perform search is interesting and novel as far as I know.\n\n2. Empirical performance on two important problems TSP and MIS is impressive.\n\nWEAKNESSES:\n1. While the paper is interesting, a weakness is that the authors did not clearly state what assumptions they made, or the proposed approach is suitable in solving which kinds of problems. For example:\n\na. The decision variables are binary. Though it is possible to transform non-binary problems into binary ones, a large number of variables could be involved, making the proposed approach impractical. \n\nb. The constraints can be transformed to some sort of penalty with meaningful gradient.\n\nc. A problem-specific post-processing procedure is needed to transform the prediction into feasible solutions.\n\nThe authors only evaluate the proposed method on two simple problems TSP and MIS, which satisfies the above conditions. But it is important to demonstrate its effectiveness on more challenging and practical problems. \n\n2. The distribution learning method in Section 3.2 heavily relies on [19]. The proof of Proposition 1 is adapted from [39]. The authors should be precise about what are the novel techniques and which parts are borrowed from existing works. \n\n3. Some design and implementation details lacks explanation. Please see the below questions.\n\nQUESTIONS:\n1. What kind of problems can the proposed approach solve? Specifically, what kind of variables and constraints, and is it easy to design a problem-specific post-processing procedure?\n\n2. How is the diffusion model in Section 3.2 different from the one in Difusco [19]?\n\n3. Some questions about design and implementation:\n\na. For TSP, why and how is 2OPT applied? It looks like 2OPT is independent from the proposed gradient search mechanism. In this case, it is a general solution improvement technique and should be applied to other baselines for fair comparison. Moreover, better solution quality with 2OPT is not suprising and does not help in justifying the contribution of the proposed method.\n\nb. The authors state that \"For gradient search, T2TCO generally involves 3 iterations with 10 guided denoising steps for each\". Why you need 3 iterations and how about the performance of only 1 iteration? How do you perform the 3 iterations, independntly and then picking the best solution?\n\nc. The authors used k=50 for graph sparsification. How sensitive does the proposed method w.r.t. to k?\n\nLIMITATIONS:\nAs mentiond above, this paper lacks a discussion on the applicability of the proposed approach, i.e. it is suitable for what kind of problems. The authors also did not explicitly describe the limitations of the proposed method.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a training to testing (T2TCO) framework to better align with the goal of combinatorial optimization (CO) problems, instead of pursuing the average expected performance, Real-world CO problems care more about instance-wise optimal solution. And thus, the paper first models the solution distribution with generative diffusion model and further conducts gradient search during testing to fine-tune the results. The framework design is novel, interesting, and sound. Experiments show the significant superiority of the framework.\n\nSTRENGTHS:\nThe paper is well written. The motivation of the paper aims towards the goal of per-instance optimality for CO problems and introduces a novel T2TCO framework to resolve the problems. The gradient-search on each testing instance requires no longer fine-tuning the parameters of the model, which is also an interesting improvement.\nThe paper conducts experiments on two representative CO problems and both achieves better results than the state-of-the-art methods.\n\nWEAKNESSES:\nThe only concern I could think about is the cross-domain generalization ability or even the in-distribution generalization ability of the supervised learning framework for CO problems.\n\nQUESTIONS:\n1.\tIs there any comparison on the training resource requirement / training time between T2TCO and other state-of-the-art methods listed in the paper? Could the authors provide some experiment data on that to give the readers a feeling about the training difficulty of the framework? (e.g. How many GPU/CPU hours, how much space, how many data does T2TCO require to converge, how about the other methods listed in the paper?)\n\nLIMITATIONS:\nThe motivation is practical, the framework is novel, interesting and the methodology is sound. Therefore, I omit the limitation of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents T2TCO, a new approach for neural combinatorial optimization based on using gradient-based search in the solution  space during test time. The proposed approach leverages generative modeling (specifically diffusion) for graduated solution improvement but employs an additional energy-based model (EBM) to evaluate the compatibility of solutions with the optimal solution. Experiments show improvement over DIFUSCO (similar generative modeling approach but without test-time gradient-based search) and a range of baselines in two popular CO benchmarks: TSP and MIS.\n\nSTRENGTHS:\n- Interesting work suggesting a new approach for ML4CO. The approach extends DIFUSCO (or similar approaches) with gradient-based search during test time to improve performance.\n- Experiments show the proposed approach outperforms the baselines. In particularly, it outperforms the recent DIFUSCO which it extends with gradient-based test-time search.\n\nWEAKNESSES:\n- Related literature: there are several relevant works that are not being mentioned and discussed. First, gradient-based search in prediction/inference (test-time) for different purposes has been proposed before in a range of settings. e.g., in [1], [2], [3]. While using it in combinatorial optimization is new (to my knowledge), the principles have been proposed before and related previous works should be mentioned and similarities and differences highlighted. Second, using diversity-focused solution generation (test-time) in combinatorial problems has been studied before e.g., in [4] and [5]. Third, introducing constraints to gradient-based search by continuous relaxations has been done before, e.g., in [6] and [7].\n\t* related to the above, it is important to mark what parts of the work is based on previous work. In particular, it sounds like Section 3.2 is primarily based on DIFUSCO, which if indeed is the case, should be clearly stated.\n\n- Experiments: (1) No experiments on generalization: it sounds like each group of instances (e.g., TSP-100) was trained and tested on instances from the same size. Previous work on ML4CO has often looked at the ability of networks trained on smaller sizes to generalize well for larger problems (including in DIFUSCO [8]). It is not clear how the proposed gradient-based search approach works in such settings. (2) While experiments show consistent improvement over previous SOTA (usually DIFUSCO), the improvement is often somewhat modest (except for MIS with ER graphs) and there is no clear analysis to demonstrate where the gain are comping from (e.g., is it consistent but modest gain across all instances vs. a small number of instances that got significant boost).\n\n- The proposed method requires manual relaxation of constraints to penalty terms and therefore is not clearly generic to all CO problems (and it is not clear that it will work well across relaxations). This is not a critical issue but it would be interesting to cover a set of popular types of constraints and their relaxations and make sure they are tested. Potentially, this can be based on previous literature (e.g., [6] where a list of continuous relaxations for popular hard constraints is provided).\n\n- Not a critical issue, but the paper seems to build extensively on DIFUSCO that, as far as I can see, is not yet been published in a peer-reviewed venue.\n\nMinor:\n- line 51: desired to estimate -> \"designed to estimate\"?\n- line 299: I think \"TSP-50\" here should be \"TSP-500\"\n- line 310: \"T2TCO is not that sensitive to the quality of labels in training\" - I think this statement is too strong given that the quality of labels seems to have significant impact on the performance (it may be that it is less sensitive than other approaches but it is still sensitive).\n\n\n[1] Belanger, D., Yang, B., & McCallum, A. (2017, July). End-to-end learning for structured prediction energy networks. In International Conference on Machine Learning (pp. 429-439). PMLR.\n\n[2] Chen, Y., Shi, Y., & Zhang, B. (2019, February). Optimal Control Via Neural Networks: A Convex Approach. In International Conference on Learning Representations.\n\n[3] Amos, B., Xu, L., & Kolter, J. Z. (2017, July). Input convex neural networks. In International Conference on Machine Learning (pp. 146-155). PMLR.\n\n[4] Cohen, E., & Beck, J. C. (2021). Heavy-Tails and Randomized Restarting Beam Search in Goal-Oriented Neural Sequence Decoding. In Integration of Constraint Programming, Artificial Intelligence, and Operations Research: 18th International Conference, CPAIOR 2021, Vienna, Austria, July 5–8, 2021, Proceedings 18 (pp. 115-132). Springer International Publishing.\n\n[5] Bai, Y., Zhao, W., & Gomes, C. P. (2021). Zero Training Overhead Portfolios for Learning to Solve Combinatorial Problems. arXiv preprint arXiv:2102.03002.\n\n[6] Chen, Di, et al. \"Deep reasoning networks for unsupervised pattern de-mixing with constraint reasoning.\" International Conference on Machine Learning. PMLR, 2020.\n\n[7] Chen, Di, et al. \"Automating crystal-structure phase mapping by combining deep learning with constraint reasoning.\" Nature Machine Intelligence 3.9 (2021): 812-822.\n\n[8] Sun, Z., & Yang, Y. (2023). Difusco: Graph-based diffusion solvers for combinatorial optimization. arXiv preprint arXiv:2302.08224.\n\nQUESTIONS:\nI would appreciate the authors' response on my main concerns listed above.\n\nLIMITATIONS:\nWhile there is no explicit discussion of limitation, I have no concerns regarding the impact of the manuscript.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}