SlowGuess/NeurIPS_2023_OCR huggingface dataset:
On small instances, our results match the ground-truth solutions and generate average gaps of  $-0.00009\%$ ,  $-0.002\%$  and  $-0.00011\%$  respectively on instances with  $n = 20, 50, 100$ , where the negative values are the results of the rounding problem. The total runtime of our method remains competitive w.r.t. all other learning baselines. On larger instances with  $n = 200, 500$  and  $1,000$ , we notice that traditional solvers (Concorde, Gurobi) fail to generate the optimal solutions within reasonable time when the size of problems grows. For RL/SL baselines, they generate results far away from ideal solutions, particularly for cases with  $n = 1,000$ . Our UTSP method is able to obtain  $0.0918\%$ ,  $0.8394\%$  and  $1.1770\%$  on TSP 200, 500 and  $1,000$ , respectively. We remark that UTSP outperforms the existing learning baselines on larger instances (TSP 200, 500, 1000) $^2$ . More discussion between Fu et al. [2021] and UTSP can be found in Appendix C.
Our model takes less training time than RL/SL method because we require very few training instances. Taking TSP 100 as an example, RL/SL needs 1 million training instances, and the total training time can take one day using a NVIDIA V100 GPU, while our method only takes about 30 minutes with 2,000 training instances. The training data size does not increase w.r.t. TSP size. Our training data consists of 2,000 instances for TSP 200, 500 and 1,000. At the same time, the UTSP model also remains very lightweight. On TSP 100, we use a 2-layer SAG with 64 hidden units and the model consists of 44,392 trainable parameters. In contrast, RL method in Kool et al. [2019] takes approximately 700,000 parameters and the SL method in Joshi et al. [2022] takes approximately 350,000 parameters.
5.3 Expressive Power of GNNs
Our UL method generalizes well to unseen examples without requiring a large number of training samples. This is because the loss function in Equation 2 is fully differentiable w.r.t. the parameters
${}^{2}$  On TSP 500,when we increase the time budget for Search,we achieve  ${0.42}\%$  in  ${1.37}\mathrm{\;m} + {8.32}\mathrm{\;m}$  .
Figure 2: TSP 100 training curve using Unsupervised Learning surrogate loss. We compare two GNN models: GCN Kipf and Welling [2016] and SAG Min et al. [2022], where GCN is a low-pass model and SAG is a low-pass + band-pass model.
map  $\mathcal{H}$ , the model learns to assign large weights to more promising edges and small weights to less promising ones through backpropagation without any prior knowledge of the ground truth or any exploration step. However, when using SL, the model learns from the TSP solutions, which fails when multiple solutions exist or the solutions are not optimal Li et al. [2018]. While for RL, the model often encounters an exploration dilemma and is not guaranteed to converge Bengio et al. [2021] Joshi et al. [2019b]. Overall, UTSP requires fewer training samples and has better generalization compared to SL/RL models.
We aim to generate a non-smooth soft indicator matrix  $\mathbb{T}$  and build an expressive heat map  $\mathcal{H}$  to guide the search algorithm. However, most GNNs aggregate information from adjacent nodes and these aggregation steps usually consist of local averaging operations, which can be interpreted as a low-pass filter and causes the oversmoothing problem Wenkel et al. [2022]. The low-pass model generates a smooth soft indicator matrix  $\mathbb{T}$ , which finally makes the elements in  $\mathcal{H}$  become indistinguishable. So it becomes difficult to discriminate whether the edges belong to the optimal solution or not. In our model, we assume all nodes in the graph are connected, so every node has  $n - 1$  connected to neighbouring nodes. This means every node receives messages from all other nodes and we have a global averaging operation over the graph, this can lead to severe oversmoothing issue.
To avoid oversmoothing, one solution is to use shallow GNNs. However, this would result in narrow receptive fields and create the problem of underreaching Barceló et al. [2020]. Our model uses SAG because this scattering-based method helps overcome the oversmoothing problem by combining band-pass wavelet filters with GCN-type filters Min et al. [2022]. Figure 2 illustrates the training loss on TSP 100 and the differences between our SAG model and the graph convolutional network (GCN) Kipf and Welling [2016], where GCN only performs low-pass filtering on graph signals Nt and Maehara [2019]. When using GCN, the training loss decreases slowly, and the validation loss reaches a plateau after we train the model for 20 epochs. This is because the low-pass model generates a smooth  $\mathbb{T}$ . Such a smooth  $\mathbb{T}$  results in an indistinguishable  $\mathcal{H}$ , detrimentally impacting the training process. Instead, we observe lower training and validation loss when using SAG; this suggests that SAG generates a more expressive representation which facilitates the training process.
Figure 3 illustrates the generated heat maps using GCN and SAG on a TSP 100 instance, we choose this instance from the validation set randomly. When using the GCN, due to the oversmoothing problem, the model generates a smooth representation and  $\mathcal{H}$  becomes indistinguishable. The elements in  $\mathcal{H}$  have a small variance and most of them are  $\sim 0.01$ . Instead, the SAG generates a discriminative representation and the elements in the heat map have a larger variance.
in SAG and we are able to train the model in an end-to-end fashion. In other words, given a heat
Figure 3: Left: The heat map  $\mathcal{H}$  generated using GCN on TSP 100. The diagonal elements are set to 0.  $X$ -axis and  $y$ -axis are the city indices, right: The heat map  $\mathcal{H}$  generated using SAG on TSP 100. The diagonal elements are set to 0.  $X$ -axis and  $y$ -axis are the city indices.
Here, we train both GCN and SAG with the same loss function. So the differences illustrated in Figure 3 are the direct result of overcoming the oversmoothing problem.
6 Search Space Reduction
To understand what happens during our training process, we study how the prediction edge set  $\Pi$  changes with training time. As mentioned, let  $\Pi$  denote undirected edge set in  $\mathcal{H}'$ , and let  $\Gamma$  denote the ground truth edge set,  $\eta = |\Gamma \cap \Pi| / |\Gamma|$  is the extent of how good our prediction set  $\Pi$  covers the solution  $\Gamma$ . If  $\eta = 1$ , then  $\Gamma$  is a subset of  $\Pi$ , which means our prediction edge set successfully covers all ground truth edges. Similarly,  $\eta = 0.95$  means we cover  $95\%$  ground truth edges.
Figure 4: Left: Average edge overlap coefficient  $\eta$  w.r.t. training epochs using SAG and GCN on TSP 100 ( $M = 10$ ), right: Number of fully covered instances w.r.t. training epochs using SAG and GCN on TSP 100. The validation set consists of 1,000 samples ( $M = 10$ ).
Figure 4 shows how the average overlap coefficient  $\eta$  changes with training epochs. We calculate the coefficient based on 1,000 validation instances in TSP 100. We notice that the coefficient quickly increases to  $\sim 98\%$  after we train SAG for 10 epochs. This suggests that the surrogate loss successfully encourages the SAG to put more weights on the more promising edges. We also compare the performance with GCN. Since the loss does not decrease significantly during our training when using GCN (shown in Figure 2), it is not surprising to see the average overlap coefficient of GCN always maintains at a relatively low level. After training the model for 100 epochs, SAG model has an average coefficient of  $99.756\%$  while GCN only has  $33.893\%$ .
We then study the number of cases where our prediction edge set  $\Pi$  covers the ground truth solution. Figure 4 (right) illustrates how the number of fully covered instances  $(\eta = 1)$  changes with time.
After training the model for 100 epochs, we observe 780 fully covered instances in 1,000 validation samples using SAG while 0 instances using GCN. Finally, we calculate the average of size  $|\Pi|$ . Our results show that SAG has an average size of 583.134 edges, while for GCN, the number is 738.739.
These results also indicate a correspondence between the loss and the quality of our prediction. In most SL tasks such as classification or regression tasks, a smaller validation loss usually means we achieve better performance and the minimum of the loss corresponds to the optimal solution (100% accuracy). However, there is no theoretical guarantee that our loss in Equation 2 also measures the solution quality. Our empirical results demonstrate that a lower surrogate loss encourages the model to assign larger weights on the promising edges and reduces the search space. This implies that we can assess the quality of the generated heat maps using our loss in Equation 2.
Overall, the UL training reduces the search space from 4950 edges to 583.134 edges with over  $99\%$  overlap accuracy on average. This helps explain why our search algorithm is able to perform well within reasonable search time.
In this paper, we propose UTSP, an Unsupervised Learning method to solve the TSP. We build a surrogate loss that encourages the GNN to find the shortest path and satisfy the constraint that the path should be a Hamiltonian Cycle. The surrogate loss function does not rely on any labelled ground truth solution and helps alleviate sparse reward problems in RL. UTSP uses a two-phase strategy. We first build a heat map based on the GNN's output. The heat map is then fed into a search algorithm. Compared with RL/SL, our method vastly reduces training cost and takes fewer training samples. We further show that our UL training helps reduce the search space. This helps explain why the generated heat maps can guide the search algorithm. On the model side, our results indicate that a low-pass GNN will produce an indistinguishable representation due to the oversmoothing issue, which results in unfavorable heat maps and fails to reduce the search space. Instead, after incorporating band-pass operators into GNN, we can build efficient heat maps that successfully reduce search space. Our findings show that the expressive power of GNNs is critical for generating a non-smooth representation that helps find the solution.
In conclusion, UTSP is competitive with or outperforms other learning-based TSP heuristics in terms of solution quality and running speed. In addition, UTSP takes  $\sim 10\%$  of the number of parameters and  $\sim 0.2\%$  of (unlabelled) training samples, compared with RL or SL methods. Our UTSP framework demonstrates that by providing a surrogate loss and a GNN which encourages a non-smooth representation, we can learn the hidden patterns in TSP instances without supervision and further reduce the search space. This allows us to build a heuristic by exploiting a small amount of unlabelled data. Future directions include designing more expressive GNNs (such as adding edge features) and using different surrogate loss functions. We anticipate that these concepts will extend to more combinatorial problems.
This project is partially supported by the Eric and Wendy Schmidt AI in Science Postdoctoral Fellowship, a Schmidt Futures program; the National Science Foundation (NSF) and the National Institute of Food and Agriculture (NIFA); the Air Force Office of Scientific Research (AFOSR); the Department of Energy; and the Toyota Research Institute (TRI).
David Applegate, Ribert Bixby, Vasek Chvatal, and William Cook. Concorde tsp solver, 2006.
Keld Helsgaun. An effective implementation of the lin-kernighan traveling salesman heuristic. European journal of operational research, 126(1):106-130, 2000.
Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, Iljoo Yoon, Youngjune Gwon, and Seungjai Min. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems, 33:21188-21198, 2020.
Hoang Nt and Takanori Maehara. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550, 2019.
James Joseph Sylvester. The Collected Mathematical Papers of James Joseph Sylvester..., volume 3. University Press, 1909.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
A.1 Definition of heat map
We can write  $\mathcal{H}$  as:
$$
\mathcal {H} = \mathbb {T} \mathbb {V} \mathbb {T} ^ {T}, \tag {3}
$$
$$
\mathbb {V} = \left( \begin{array}{c c c c c c c c} 0 & 1 & 0 & 0 & \dots & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & \dots & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 & \dots & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \ddots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \ddots & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & \dots & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & \dots & 0 & 0 & 1 \\ 1 & 0 & 0 & 0 & \dots & 0 & 0 & 0 \end{array} \right)
$$
is the Sylvester shift matrix Sylvester [1909],  $\mathbb{V} \in \mathbb{R}^{n \times n}$ . We can interpret  $\mathbb{V}$  as a cyclic permutation operator that performs a circular shift.
A.2 Unsupervised Loss
We can also write Equation 2 in a more compact form:
$$
\mathcal {L} = \lambda_ {1} \underbrace {\sum_ {i = 1} ^ {n} \left(\sum_ {j = 1} ^ {n} \mathbb {T} _ {i , j} - 1\right) ^ {2}} _ {\text {R o w - w i s e c o n s t r a i n t}} + \underbrace {\sum_ {i = 1} ^ {n} \sum_ {j = 1} ^ {n} \tilde {\mathbf {D}} _ {i , j} \mathcal {H} _ {i , j}} _ {\text {M i n i m i z e t h e d i s t a n c e}}, \tag {4}
$$
where  $\tilde{\mathbf{D}} = \mathbf{D} + \lambda_2\mathcal{I}_n,\mathcal{I}_n\in \mathbb{R}^{n\times n}$  is the identity matrix.
Table 4: Search parameters for all the TSP experiments.
α	β	M	K	T
TSP-20	0	10	8	10	60
TSP-50	0	10	8	[5, 15)	150
TSP-100	0	10	8	[5, 35)	300
TSP-200	0	10	8	[10, 90)	600
TSP-500	0	50	5	[30, 130)	1000
TSP-1000	0	50	5	[10, 110)	2000
B Training and Search Details
We train our model using Adam Kingma and Ba [2014]. All models are trained using Nvidia V100 GPU. All the search-related parameters are listed in Table 4.  $M$  is the size of the candidate set of each city.  $K$  is the maximal number of edges we can remove in one action, and for each round of local search, we randomly select one number from the listed interval.  $T$  is the total number of actions we will try to expand one node. Here, we set  $\alpha = 0$  to show that our unsupervised model generates an informative heat map. Lower  $\alpha$  means the local search algorithm focuses more on the edges with higher heat map value. Actually, in the experiments, we find the results are similar with  $\alpha \leq 1$ .
C Running Time Discussion
As discussed in Kool et al. [2019], running time is important but hard to compare since it is affected by many factors. In the table 1, we report the time for solving all the test instances.
For the UTSP (our method) and the state-of-the-art learning-based method Att-GCRN Fu et al. [2021], we run the search algorithm on exactly the same environment (one Intel Xeon Gold 6326) for a fair comparison. And for other baselines, we refer to the results from Fu et al. [2021]. So, the time there is only for indicative purposes since the computing hardware is different.
Lemma D.1. Let  $q_{i}$  denote the row index of the non-zero element in  $i$ -th column in  $\mathbb{T}$ ,  $\mathbb{T}_{q_i,i} = 1$ ,  $q_{i} \in \{1,2,3,4,\dots,n\}$ . When each row and column in  $\mathbb{T}$  have one value 1 (True) and  $n - 1$  value 0 (False), then  $q_{i} = q_{j}$  if and only if  $i = j$ .
Proof. If there exist a  $(i,j)$  pair where  $q_{i} = q_{j}$  when  $i\neq j$ , then  $\mathbb{T}_{q_i,i} = 1$  and  $\mathbb{T}_{q_j,j} = \mathbb{T}_{q_i,j} = 1$ . This means  $q_{i}$ -th row has two non-zero elements, which leads to a contradiction.
Lemma D.2. Consider graph  $\mathcal{G}$  with  $n$  nodes, for any  $\mathbb{T} \in \mathbb{R}^{n \times n}$  with  $\mathbb{T}_{i,j} \geq 0$ , when each row and column in  $\mathbb{T}$  have one value 1 (True) and  $n - 1$  value 0 (False). Then, each row and column in  $\mathcal{H}$  also have one value 1 (True) and  $n - 1$  value 0 (False), which means each city corresponds to only one beginning point and one ending point.
Proof. First, it is clear that for  $\forall a, b, \mathcal{H}_{a,b} \in \mathbb{Z}_{\geq 0}$ . Let's assume  $\mathbb{T}_{a,l} = \mathbb{T}_{b,m} = 1 (a \neq b, l \neq m)$ , since  $\mathcal{H}_{i,j} = \sum_{k=1}^{n} \mathbb{T}_{i,k} \mathbb{T}_{j,k+1(\mathrm{mod} n)}$ , we then have
$$
\begin{array}{l} \sum_ {j = 1} ^ {n} \mathcal {H} _ {a, j} = \sum_ {j = 1} ^ {n} \sum_ {k = 1} ^ {n} \mathbb {T} _ {a, k} \mathbb {T} _ {j, k + 1 (\mathrm {m o d} n)} \\ = \sum_ {k = 1} ^ {n} \mathbb {T} _ {a, k} \left\{\sum_ {j = 1} ^ {n} \mathbb {T} _ {j, k + 1 (\mathrm {m o d} n)} \right\} \\ = \sum_ {k = 1} ^ {n} \mathbb {T} _ {a, k} = 1. \\ \end{array}
$$
This implies that the summation of each row in  $\mathcal{H}$  is 1. Similarly,
$$
\begin{array}{l} \sum_ {i = 1} ^ {n} \mathcal {H} _ {i, b} = \sum_ {i = 1} ^ {n} \sum_ {k = 1} ^ {n} \mathbb {T} _ {i, k} \mathbb {T} _ {b, k + 1 (\mathrm {m o d} n)} \\ = \sum_ {k = 1} ^ {n} \mathbb {T} _ {b, k + 1 (\mathrm {m o d} n)} \left\{\sum_ {i = 1} ^ {n} \mathbb {T} _ {i, k} \right\} \\ = \sum_ {k = 1} ^ {n} \mathbb {T} _ {b, k + 1 (\mathrm {m o d} n)} = 1. \\ \end{array}
$$
This suggests the summation of each column in  $\mathcal{H}$  is 1. Since each element in  $\mathcal{H}_{i,j} \in \mathbb{Z}_{\geq 0}$ , we can then conclude that each row and column in  $\mathcal{H}$  have one value 1 (True) and  $n - 1$  value 0 (False). Also,  $\mathcal{H}_{ii} = \sum_{k=1}^{n} \mathbb{T}_{i,k+1 (\bmod n)} \mathbb{T}_{i,k} = 0$ , this indicates that the elements in the main diagonal of  $\mathcal{H}$  are 0, which implies no self-loops.
As mentioned, the  $i$ -th row in  $\mathcal{H}$  is the probability distribution of directed edges start from city  $i$ , and the  $j$ -th column is the probability distribution of directed edges end in city  $j$ . Because each row and column in  $\mathcal{H}$  have one value 1 (True) element and  $\mathcal{H}$ 's diagonal entries are all zero, this means that each city is the beginning point of one directed edge and is also the ending point of another different directed edge.
Lemma D.3. There is at least one cycle in  $\mathcal{H}$  which contains  $n$  edges and visits all cities when each row and column in  $\mathbb{T}$  have one value 1 (True) and  $n - 1$  value 0 (False).
Figure 5: Illustration of building a cycle from transition matrix  $\mathbb{T}$ . Here  $q_{k-1} = 3$ ,  $q_k = n - 2$ ,  $q_{k+1} = 2$ ,  $q_{n-1} = 1$ ,  $q_n = n$  and  $q_1 = 4$ , this means  $\mathcal{H}$  contains the following four directed edges:  $3 \to n - 2$ ,  $n - 2 \to 2$ ,  $1 \to n$  and  $n \to 4$ .
Proof. Lemma D.1 indicates that  $q_{i} \neq q_{j}$  when  $i \neq j$  and  $q_{i} \in \{1, 2, 3, 4, \dots, n\}$ , then  $\cup_{i=1}^{n} q_{i} = \{1, 2, 3, 4, \dots, n\}$ . From Equation 3, we have
$$
\begin{array}{l} \mathcal {H} _ {q _ {i}, q _ {i + 1}} = \sum_ {k = 1} ^ {n} \mathbb {T} _ {q _ {i}, k} \mathbb {T} _ {q _ {i + 1}, k + 1 (\mathrm {m o d} n)} \\ \geq \mathbb {T} _ {q _ {i}, i} \mathbb {T} _ {q _ {i + 1}, i + 1 (\mathrm {m o d} n)} \\ \geq 1. \\ \end{array}
$$
Using Lemma D.2, since each row and column in  $\mathcal{H}$  have one value 1 (True) and  $n - 1$  value 0 (False), it suffices to show that  $1 \geq \mathcal{H}_{q_i,q_{i + 1}} \geq 1$ , therefore  $\mathcal{H}_{q_i,q_{i + 1}} = 1$ . This suggests that there is a directed edge from city  $q_{i}$  to  $q_{i + 1}$ . We can then construct a cycle  $\mathcal{C}$  from  $\mathcal{H}$ , we can write  $\mathcal{C}$  as
$$
q _ {1} \rightarrow q _ {2} \rightarrow q _ {3} \rightarrow q _ {4} \rightarrow \dots \rightarrow q _ {n} \rightarrow q _ {1},
$$
where  $\rightarrow$  is a directed edge. Since  $\cup_{i=1}^{n} q_i = \{1, 2, 3, 4, \dots, n\}$ , cycle  $\mathcal{C}$  visits all  $n$  cities and have  $n$  edges. One example of how to build  $\mathcal{H}_{q_i, q_{i+1}}$  from  $\mathbb{T}$  is shown in Figure 5.
Corollary D.4.  $\mathcal{H}$  represents one Hamiltonian Cycle when each row and column in  $\mathbb{T}$  have one value 1 (True) and  $n - 1$  value 0 (False).
Proof. From Lemma D.3, cycle  $\mathcal{C}$  contains  $n$  edges and visits all cities, if there exists another edge  $(i,j)$  which does not belong to  $\mathcal{C}$ , then city  $i$  is the starting point of at least two edges and city  $j$  is the ending point of at least two edges. This results in a contradiction with Lemma D.2. Thus, it suffices to conclude that  $\mathcal{C}$  visits each city exactly once and  $\mathcal{H}$  only contains the edges in  $\mathcal{C}$ . This implies that  $\mathcal{H}$  represents one Hamiltonian Cycle.