SlowGuess/NeurIPS_2023_OCR huggingface dataset:
After the training of the parameterized denoising network according to Eq. 4, the solutions are sampled from the diffusion models  $p_{\theta}(\mathbf{x}_0|s)$  for final evaluation. However, probabilistic generative models such as DIFUSCO cannot guarantee that the sampled solutions are feasible according to the definition of CO problems. Therefore, specialized decoding strategies are designed for the two CO problems studied in this paper.
Heatmap Generation The diffusion models  $p_{\theta}(\cdot |s)$  produce discrete variables  $\mathbf{x}$  as the final predictions by applying Bernoulli sampling (Eq. 6) for discrete diffusion or quantization for continuous diffusion. However, this process discards the comparative information that reflects the confidence of the predicted variables, which is crucial for resolving conflicts in the decoding process. To preserve this information, we adapt the diffusion models to generate heatmaps [53, 92] by making the following appropriate modifications: 1) For discrete diffusion, the final score of  $p_{\theta}(\mathbf{x_0} = 1|s)$  is preserved as the heatmap scores; 2) For continuous diffusion, we remove the final quantization and use  $0.5(\hat{\mathbf{x}}_0 + 1)$  as the heatmap scores. Note that different from previous heatmap approaches [53, 92] that produce a single conditionally independent distribution for all variables, DIFUSCO can produce diverse multimodal output distribution by using different random seeds.
TSP Decoding Let  $\{A_{ij}\}$  be the heatmap scores generated by DIFUSCO denoting the confidence of each edge. We evaluate two approaches as the decoding method following previous work [32, 92]: 1) Greedy decoding [32], where all the edges are ranked by  $(A_{ij} + A_{ji}) / \| \mathbf{c}_i - \mathbf{c}_j\|$ , and are inserted into the partial solution if there are no conflicts. 2-opt heuristics [71] are optionally applied. 2) Monte Carlo Tree Search (MCTS) [27], where  $k$ -opt transformation actions are sampled guided by the heatmap scores to improve the current solutions. Due to the space limit, a detailed description of two decoding strategies can be found in the appendix.
MIS Decoding Let  $\{a_i\}$  be the heatmap scores generated by DIFUSCO denoting the confidence of each node. A greedy decoding strategy is used for the MIS problem, where the nodes are ranked by  $a_i$  and inserted into the partial solution if there are no conflicts. Recent research [8] pointed out that the graph reduction and 2-opt search [2] can find near-optimal solutions even starting from a randomly generated solution, so we do not use any post-processing for the greedy-decoded solutions.
Solution Sampling A common practice for probabilistic CO solvers [64] is to sample multiple solutions and report the best one. For DIFUSCO, we follow this practice by sampling multiple heatmaps from  $p_{\theta}(\mathbf{x}_0|s)$  with different random seeds and then applying the greedy decoding algorithm described above to each heatmap.
4 Experiments with TSP
We use 2-D Euclidean TSP instances to test our models. We generate these instances by randomly sampling nodes from a uniform distribution over the unit square. We use TSP-50 (with 50 nodes) as the main benchmark to compare different model configurations. We also evaluate our method on larger TSP instances with 100, 500, 1000, and 10000 nodes to demonstrate its scalability and performance against other state-of-the-art methods.
Datasets We generate and label the training instances using the Concorde exact solver [3] for TSP-50/100 and the LKH-3 heuristic solver [39] for TSP-500/1000/10000. We use the same test instances as [54, 64] for TSP-50/100 and [27] for TSP-500/1000/10000.
Graph Sparsification We use sparse graphs for large-scale TSP problems to reduce the computational complexity. We sparsify the graphs by limiting each node to have only  $k$  edges to its nearest neighbors based on the Euclidean distances. We set  $k$  to 50 for TSP-500 and 100 for TSP-1000/10000. This way, we avoid the quadratic growth of edges in dense graphs as the number of nodes increases.
Model Settings  $T = 1000$  denoising steps are used for the training of DIFUSCO on all datasets. Following Ho et al. [40], Graikos et al. [32], we use a simple linear noise schedule for  $\{\beta_t\}_{t=1}^T$ , where  $\beta_1 = 10^{-4}$  and  $\beta_T = 0.02$ . We follow Graikos et al. [32] and use the Greedy decoding + 2-opt scheme (Sec. 3.5) as the default decoding scheme for experiments.
Evaluation Metrics In order to compare the performance of different models, we present three metrics: average tour length (Length), average relative performance gap (Gap), and total run time (Time). The detailed description can be found in the appendix.
Figure 1: Comparison of continuous (Gaussian noise) and discrete (Bernoulli noise) diffusion models with different inference diffusion steps and inference schedule (linear v.s. cosine).
Table 1: Comparing results on TSP-50 and TSP-100. * denotes the baseline for computing the performance gap. † indicates that the diffusion model samples a single solution as its greedy decoding scheme. Please refer to Sec. 4 for details.
ALGORITHM	TYPE	TSP-50	TSP-100
LENGTH↓	GAP(%)↓	LENGTH↓	GAP(%)↓
CONCORDE*	EXACT	5.69	0.00	7.76	0.00
2-OPT	HEURISTICS	5.86	2.95	8.03	3.54
AM	GREEDY	5.80	1.76	8.12	4.53
GCN	GREEDY	5.87	3.10	8.41	8.38
TRANSFORMER	GREEDY	5.71	0.31	7.88	1.42
POMO	GREEDY	5.73	0.64	7.84	1.07
SYM-NCO	GREEDY	-	-	7.84	0.94
DPDP	1k-IMPROVEMENTS	5.70	0.14	7.89	1.62
IMAGE DIFFUSION	GREEDY†	5.76	1.23	7.92	2.11
OURS	GREEDY†	5.70	0.10	7.78	0.24
AM	1k×SAMPLING	5.73	0.52	7.94	2.26
GCN	2k×SAMPLING	5.70	0.01	7.87	1.39
TRANSFORMER	2k×SAMPLING	5.69	0.00	7.76	0.39
POMO	8×AUGMENT	5.69	0.03	7.77	0.14
SYM-NCO	100×SAMPLING	-	-	7.79	0.39
MDAM	50×SAMPLING	5.70	0.03	7.79	0.38
DPDP	100k-IMPROVEMENTS	5.70	0.00	7.77	0.00
OURS	16×SAMPLING	5.69	-0.01	7.76	-0.01
(a) Continuous diffusion
(b) Discrete diffusion
Figure 2: The performance Gap  $(\%)$  are shown for continuous diffusion (a) and discrete diffusion (b) models on TSP-50 with different diffusion steps and number of samples. Their corresponding per-instance run-time (sec) are shown in (c), where the decomposed analysis (neural network + greedy decoding + 2-opt) can be found in the appendix.
(c) Runtime
4.2 Design Analysis
Discrete Diffusion v.s. Continuous Diffusion We first investigate the suitability of two diffusion approaches for combinatorial optimization, namely continuous diffusion with Gaussian noise and discrete diffusion with Bernoulli noise (Sec. 3.2). Additionally, we explore the effectiveness of different denoising schedules, such as linear and cosine schedules (Sec. 3.3), on CO problems. To efficiently evaluate these model choices, we utilize the TSP-50 benchmark.
Note that although all the diffusion models are trained with a  $T = 1000$  noise schedule, the inference schedule can be shorter than  $T$ , as described in Sec. 3.3. Specifically, we are interested in diffusion models with 1, 2, 5, 10, 20, 50, 100, and 200 diffusion steps.
Fig. 1 demonstrates the performance of two types of diffusion models with two types of inference schedules and various diffusion steps. We can see that discrete diffusion consistently outperforms the continuous diffusion models by a large margin when there are more than 5 diffusion steps². Besides, the cosine schedule is superior to linear on discrete diffusion and performs similarly on continuous diffusion. Therefore, we use cosine for the rest of the paper.
More Diffusion Iterations vs. More Sampling By utilizing effective denoising schedules, diffusion models are able to adaptively infer based on the available computation budget by predetermining the total number of diffusion steps. This is similar to changing the number of samples in previous probabilistic neural NPC solvers [64]. Therefore, we investigate the trade-off between the number of diffusion iterations and the number of samples for diffusion-based NPC solvers.
2We also observe similar patterns on TSP-100, where the results are reported in the appendix.
Table 2: Results on large-scale TSP problems. RL, SL, AS, G, S, BS, and MCTS denotes Reinforcement Learning, Supervised Learning, Active Search, Greedy decoding, Sampling decoding, Beam-search, and Monte Carlo Tree Search, respectively. * indicates the baseline for computing the performance gap. Results of baselines are taken from Fu et al. [27] and Qiu et al. [92], so the runtime may not be directly comparable. See Section 4 and appendix for detailed descriptions.
ALGORITHM	TYPE	TSP-500	TSP-1000	TSP-10000
LENGTH↓	GAP↓	TIME↓	LENGTH↓	GAP↓	TIME↓	LENGTH↓	GAP↓	TIME↓
CONCORDE	EXACT	16.55*	—	37.66m	23.12*	—	6.65h	N/A	N/A	N/A
GUROBI	EXACT	16.55	0.00%	45.63h	N/A	N/A	N/A	N/A	N/A	N/A
LKH-3 (DEFAULT)	HEURISTICS	16.55	0.00%	46.28m	23.12	0.00%	2.57h	71.77*	—	8.8h
LKH-3 (LESS TRAILS)	HEURISTICS	16.55	0.00%	3.03m	23.12	0.00%	7.73m	71.79	—	51.27m
FARTHEST INSERTION	HEURISTICS	18.30	10.57%	0s	25.72	11.25%	0s	80.59	12.29%	6s
AM	RL+G	20.02	20.99%	1.51m	31.15	34.75%	3.18m	141.68	97.39%	5.99m
GCN	SL+G	29.72	79.61%	6.67m	48.62	110.29%	28.52m	N/A	N/A	N/A
POMO+EAS-EMB	RL+AS+G	19.24	16.25%	12.80h	N/A	N/A	N/A	N/A	N/A	N/A
POMO+EAS-TAB	RL+AS+G	24.54	48.22%	11.61h	49.56	114.36%	63.45h	N/A	N/A	N/A
DIMES	RL+G	18.93	14.38%	0.97m	26.58	14.97%	2.08m	86.44	20.44%	4.65m
DIMES	RL+AS+G	17.81	7.61%	2.10h	24.91	7.74%	4.49h	80.45	12.09%	3.07h
OURS (DIFUSCO)	SL+G†	18.35	10.85%	3.61m	26.14	13.06%	11.86m	98.15	36.75%	28.51m
OURS (DIFUSCO)	SL+G†+2-OPT	16.80	1.49%	3.65m	23.56	1.90%	12.06m	73.99	3.10%	35.38m
EAN	RL+S+2-OPT	23.75	43.57%	57.76m	47.73	106.46%	5.39h	N/A	N/A	N/A
AM	RL+BS	19.53	18.03%	21.99m	29.90	29.23%	1.64h	129.40	80.28%	1.81h
GCN	SL+BS	30.37	83.55%	38.02m	51.26	121.73%	51.67m	N/A	N/A	N/A
DIMES	RL+S	18.84	13.84%	1.06m	26.36	14.01%	2.38m	85.75	19.48%	4.80m
DIMES	RL+AS+S	17.80	7.55%	2.11h	24.89	7.70%	4.53h	80.42	12.05%	3.12h
OURS (DIFUSCO)	SL+S	17.23	4.08%	11.02m	25.19	8.95%	46.08m	95.52	33.09%	6.59h
OURS (DIFUSCO)	SL+S+2-OPT	16.65	0.57%	11.46m	23.45	1.43%	48.09m	73.89	2.95%	6.72h
ATT-GCN	SL+MCTS	16.97	2.54%	2.20m	23.86	3.22%	4.10m	74.93	4.39%	21.49m
DIMES	RL+MCTS	16.87	1.93%	2.92m	23.73	2.64%	6.87m	74.63	3.98%	29.83m
DIMES	RL+AS+MCTS	16.84	1.76%	2.15h	23.69	2.46%	4.62h	74.06	3.19%	3.57h
OURS (DIFUSCO)	SL+MCTS	16.63	0.46%	10.13m	23.39	1.17%	24.47m	73.62	2.58%	47.36m
Fig. 2 shows the results of continuous diffusion and discrete diffusion with various diffusion steps and number of parallel sampling, as well as their corresponding total run-time. The cosine denoising schedule is used for fast inference. Again, we find that discrete diffusion outperforms continuous diffusion across various settings. Besides, we find performing more diffusion iterations is generally more effective than sampling more solutions, even when the former uses less computation. For example, 20 (diffusion steps)  $\times 4$  (samples) performs competitive to 1 (diffusion steps)  $\times 1024$  (samples), while the runtime of the former is  $18.5 \times$  less than the latter.
In general, we find that 50 (diffusion steps)  $\times 1$  (samples) policy and 10 (diffusion steps)  $\times 16$  (samples) policy make a good balance between exploration and exploitation for discrete DIFUSCO models and use them as the Greedy and Sampling strategies for the rest of the experiments.
Comparison to SOTA Methods We compare discrete DIFUSCO to other state-of-the-art neural NPC solvers on TSP problems across various scales. Due to the space limit, the description of other baseline models can be found in the appendix.
Tab. 1 compare discrete DIFUSCO with other models on TSP-50 and TSP-100, where DIFUSCO achieves the state-of-the-art performance in both greedy and sampling settings for probabilistic solvers.
Tab. 2 compare discrete DIFUSCO with other models on the larger-scale TSP-500, TSP-1000, and TSP-10000 problems. Most previous probabilistic solvers (except DIMES [92]) becomes untrainable on TSP problems of these scales, so the results of these models are reported with TSP-100 trained models. The results are reported with greedy, sampling, and MCTS decoding strategies, respectively. For fair comparisons [27, 92], MCTS decoding for TSP is always evaluated with only one sampled heatmap. From the table, we can see that DIFUSCO strongly outperforms the previous neural solvers on all three settings. In particular, with MCTS-based decoding, DIFUSCO significantly improving the performance gap between ground-truth and neural solvers from  $1.76\%$  to  $0.46\%$  on TSP-500, from  $2.46\%$  to  $1.17\%$  on TSP-1000, and from  $3.19\%$  to  $2.58\%$  on TSP-10000.
Figure 3: Generalization tests of DIFUSCO trained and evaluated on TSP problems across various scales. The performance Gap  $(\%)$  with greedy decoding and 2-opt is reported.
Table 3: Results on MIS problems. * indicates the baseline for computing the optimality gap. RL, SL, G, S, and TS denote Reinforcement Learning, Supervised Learning, Greedy decoding, Sampling decoding, and Tree Search, respectively. Please refer to Sec. 5 and appendix for details.
METHOD	TYPE	SATLIB	ER-[700-800]
SIZE ↑	GAP ↓	TIME ↓	SIZE ↑	GAP ↓	TIME ↓
KAMIS	HEURISTICS	425.96*	—	37.58m	44.87*	—	52.13m
GUROBI	EXACT	425.95	0.00%	26.00m	41.38	7.78%	50.00m
INTEL	SL+G	420.66	1.48%	23.05m	34.86	22.31%	6.06m
INTEL	SL+TS	N/A	N/A	N/A	38.80	13.43%	20.00M
DGL	SL+TS	N/A	N/A	N/A	37.26	16.96%	22.71m
LWD	RL+S	422.22	0.88%	18.83m	41.17	8.25%	6.33m
DIMES	RL+G	421.24	1.11%	24.17m	38.24	14.78%	6.12m
DIMES	RL+S	423.28	0.63%	20.26m	42.06	6.26%	12.01m
OURS	SL+G	424.50	0.34%	8.76m	38.83	12.40%	8.80m
OURS	SL+S	425.13	0.21%	23.74m	41.12	8.36%	26.67m
Generalization Tests Finally, we study the generalization ability of discrete DIFUSCO trained on a set of TSP problems of a specific problem scale and evaluated on other problem scales. From Fig. 3, we can see that DIFUSCO has a strong generalization ability. In particular, the model trained with TSP-50 perform well on even TSP-1000 and TSP0-10000. This pattern is different from the bad generalization ability of RL-trained or SL-trained non-autoregressive methods as reported in previous work [54].
5 Experiments with MIS
For Maximal Independent Set (MIS), we experiment on two types of graphs that recent work [70, 1, 8, 92] shows struggles against, i.e., SATLIB [46] and Erdős-Rényi (ER) graphs [26]. The former is a set of graphs reduced from SAT instances in CNF, while the latter are random graphs. We use ER-[700-800] for evaluation, where ER- $[n-N]$  indicates the graph contains  $n$  to  $N$  nodes. Following Qiu et al. [92], the pairwise connection probability  $p$  is set to 0.15.
Datasets The training instances of labeled by the  $\mathrm{KaMIS}^3$  heuristic solver. The split of test instances on SAT datasets and the random-generated ER test graphs are taken from Qiu et al. [92].
Model Settings The training schedule is the same as the TSP solver (Sec. 4.1). For SATLIB, we use discrete diffusion with 50 (diffusion steps)  $\times 1$  (samples) policy and 50 (diffusion steps)  $\times 4$  (samples) policy as the Greedy and Sampling strategies, respectively. For ER graphs, we use continuous diffusion with 50 (diffusion steps)  $\times 1$  (samples) policy and 20 (diffusion steps)  $\times 8$  (samples) policy as the Greedy and Sampling strategies, respectively.
Evaluation Metrics We report the average size of the independent set (Size), average optimality gap (Gap), and latency time (Time). The detailed description can be found in the appendix. Notice that we disable graph reduction and 2-opt local search in all models for a fair comparison since it is pointed out by [8] that all models would perform similarly with local search post-processing.
Results and Analysis Tab. 3 compare discrete DIFUSCO with other baselines on SATLIB and ER-[700-800] benchmarks. We can see that DIFUSCO strongly outperforms previous state-of-the-art methods on SATLIB benchmark, reducing the gap between ground-truth and neural solvers from  $0.63\%$  to  $0.21\%$ . However, we also found that DIFUSCO (especially with discrete diffusion in our preliminary experiments) does not perform well on the ER-[700-800] data. We hypothesize that this is because the previous methods usually use the node-based graph neural networks such as GCN [62] or GraphSage [36] as the backbone network, while we use an edge-based Anisotropic GNN (Sec. 3.4), whose inductive bias may be not suitable for ER graphs.
<sup>3</sup>https://github.com/KarlsruheMIS/KaMIS (MIT License)
We proposed DIFUSCO, a novel graph-based diffusion model for solving NP-complete combinatorial optimization problems. We compared two variants of graph-based diffusion models: one with continuous Gaussian noise and one with discrete Bernoulli noise. We found that the discrete variant performs better than the continuous one. Moreover, we designed a cosine inference schedule that enhances the effectiveness of our model. DIFUSCO achieves state-of-the-art results on TSP and MIS problems, surpassing previous probabilistic NPC solvers in both accuracy and scalability.
For future work, we would like to explore the potential of DIFUSCO in solving a broader range of NPC problems, including Mixed Integer Programming (discussed in the appendix). We would also like to explore the use of equivariant graph neural networks [117, 45] for further improvement of the diffusion models on geometrical NP-complete combinatorial optimization problems such as Euclidean TSP. Finally, we are interested in utilizing (higher-order) accelerated inference techniques for diffusion model-based solvers, such as those inspired by the continuous time framework for discrete diffusion [12, 105].
[1] Sungsoo Ahn, Younggyo Seo, and Jinwoo Shin. Learning what to defer for maximum independent sets. In International Conference on Machine Learning, pages 134-144. PMLR, 2020.
[2] Diogo V Andrade, Maurizio GC Resende, and Renato F Werneck. Fast local search for the maximum independent set problem. Journal of Heuristics, 18(4):525-547, 2012.
[3] David Applegate, Ribert Bixby, Vasek Chvatal, and William Cook. Concorde TSP solver. https://www.math.uwaterloo.ca/tsp/concorde/index.html, 2006.
[4] Sanjeev Arora. Polynomial time approximation schemes for euclidean tsp and other geometric problems. In Proceedings of 37th Conference on Foundations of Computer Science, pages 2-11. IEEE, 1996.
[5] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981-17993, 2021.
[6] Irwan Bello, Hieu Pham, Quoc V Le, Mohammad Norouzi, and Samy Bengio. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.09940, 2016.
[7] Jieyi Bi, Yining Ma, Jiahai Wang, Zhiguang Cao, Jinbiao Chen, Yuan Sun, and Yeow Meng Chee. Learning generalizable models for vehicle routing problems via knowledge distillation. In Advances in Neural Information Processing Systems, 2022.
[8] Maximilian Böther, Otto Kißig, Martin Taraz, Sarel Cohen, Karen Seidel, and Tobias Friedrich. What's wrong with deep learning in tree search for combinatorial optimization. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=mkOHzdqY7i1.
[9] Xavier Bresson and Thomas Laurent. An experimental study of neural networks for variable graphs. 2018.
[10] Xavier Bresson and Thomas Laurent. The transformer network for the traveling salesman problem. arXiv preprint arXiv:2103.03012, 2021.
[11] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[110] Runzhong Wang, Zhigang Hua, Gan Liu, Jiayi Zhang, Junchi Yan, Feng Qi, Shuang Yang, Jun Zhou, and Xiaokang Yang. A bi-level framework for learning to solve combinatorial optimization on graphs. arXiv preprint arXiv:2106.04927, 2021.
[111] Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229-256, 1992.
[112] Lemeng Wu, Chengyue Gong, Xingchao Liu, Mao Ye, and Qiang Liu. Diffusion-based molecule generation with informative prior bridges. arXiv preprint arXiv:2209.00865, 2022.
[113] Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang, and Andrew Lim. Learning improvement heuristics for solving routing problems.. IEEE transactions on neural networks and learning systems, 2021.
[114] Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. Multi-decoder attention model with embedding glimpse for solving vehicle routing problems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12042-12049, 2021.
[115] Liang Xin, Wen Song, Zhiguang Cao, and Jie Zhang. NeuroLKH: Combining deep learning model with Lin-Kernighan-Helsgaun heuristic for solving the traveling salesman problem. Advances in Neural Information Processing Systems, 34, 2021.
[116] Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=ryGs6iA5Km.
[117] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. In International Conference on Learning Representations, 2021.
[118] Dongchao Yang, Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. Diffsound: Discrete diffusion model for text-to-sound generation. arXiv preprint arXiv:2207.09983, 2022.
[119] Emre Yolcu and Barnabás Póczos. Learning local search heuristics for boolean satisfiability. In NeurIPS, pages 7990-8001, 2019.
[120] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. Transactions on Machine Learning Research.
[121] Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming-Hsuan Yang, Yuan Hao, Irfan Essa, et al. Magvit: Masked generative video transformer. arXiv preprint arXiv:2212.05199, 2022.
[122] Cong Zhang, Wen Song, Zhiguang Cao, Jie Zhang, Puay Siew Tan, and Chi Xu. Learning to dispatch for job shop scheduling via deep reinforcement learning. arXiv preprint arXiv:2010.12367, 2020.
A Frequently Asked Questions
A.1 Does DIFUSCO rely on high-quality solutions collected in advance?
DIFUSCO does not require exact optimal solutions to train the diffusion model-based solver. For instance, in TSP-500/1000/10000, we utilize the heuristic solver LKH-3 to generate near-optimal solutions, which is nearly as fast as the neural solvers themselves (refer to Table 2 for details). Empirically (in TSP-500/1000/10000), DIFUSCO still attains state-of-the-art performance even when trained on such near-optimal solutions rather than on exact optimal ones. The training data sizes are reported in Appendix E.
A.2 This work simply applies discrete diffusion models to combinatorial optimization problems. Could you clarify the significance of this work in the field?
It is essential to highlight that our work introduces a novel approach to solving NP-complete problems using graph-based diffusion models. This approach has not been previously explored in the context of NP-complete problems, and we believe that the significant improvements in performance over the state-of-the-art on benchmark datasets emphasize the value of our work.
Drawing a parallel to the application of diffusion models in image generation and subsequent extension to videos, our work explores a similar expansion of the scope. While denoising diffusion models have been extensively researched and applied to image generation, DIFUSCO represents the first instance where these models are applied to NP-complete CO problems. Such novelty in problem formulation is fundamental for solving NP-complete problems instead of incremental ones from a methodological point of view.