SlowGuess/NeurIPS_2023_OCR huggingface dataset:
i.e.,  $o_{\lambda}^{\kappa} = h_{i_{\kappa}}$ . We intend the two streams to complement each other, coherently capturing complex correlations and dependencies among historical decisions. For the initialization, we use learnable parameters  $o_{\mu}^{1} = \hat{o}$ ,  $o_{\lambda}^{1} = \tilde{o}$ . Based on the attention between  $q_{\mu}, q_{\lambda}$  and node embeddings  $h$ , the node selection scores suggested by both streams can then be calculated:
$$
\mu_ {\kappa} = \operatorname {T a n h} \left(\left(q _ {\mu} ^ {\kappa} W _ {\mu} ^ {\text {Q u e r y}} + h W _ {\mu} ^ {\text {K e y}}\right) + \left(q _ {\mu} ^ {\kappa} W _ {\mu} ^ {\text {Q u e r y} ^ {\prime}}\right) \odot \left(h W _ {\mu} ^ {\text {K e y} ^ {\prime}}\right)\right) W _ {\mu} ^ {O}, \tag {3}
$$
$$
\lambda_ {\kappa} = \operatorname {T a n h} \left(\left(q _ {\lambda} ^ {\kappa} W _ {\lambda} ^ {\text {Q u e r y}} + h W _ {\lambda} ^ {\text {K e y}}\right) + \left(q _ {\lambda} ^ {\kappa} W _ {\lambda} ^ {\text {Q u e r y} ^ {\prime}}\right) \odot \left(h W _ {\lambda} ^ {\text {K e y} ^ {\prime}}\right)\right) W _ {\lambda} ^ {O},
$$
where all matrices  $W \in \mathbb{R}^{d \times d}$  are trainable parameters and we consider both summation and Hadamard products. Afterwards, the final categorical distribution  $P_{\theta}^{\kappa}$  is induced by:  $\text{Softmax}(C \cdot \tanh(\mu_{\kappa} + \lambda_{\kappa}))$  where  $C = 6$  and invalid choices that do not satisfy  $\Gamma[a, j_{\kappa}] \leq \Gamma[a, \kappa]$  are masked as  $-\infty$  (an E-move is enforced if all nodes are masked, which happens when the highest-ranked node was chosen in the previous I-move). A node  $x_{\kappa}$  is then sampled from  $P_{\theta}^{\kappa}$ , after which its basis move type  $\Phi_{\kappa}$  can be inferred. The decoding process continues until it early terminates or reaches the maximum limit  $K$ .
4.3 Inference with the dynamic data augmentation (D2A)
Our NeuOpt is trained using the RL algorithm tailored in our previous work [12]. During inference, our previous work [12] employed a data augmentation method that applies optimal-solution-invariant transformations (e.g., flipping coordinates) on an instance,  $\mathcal{G}$ , so as to generate a set of instances  $\{\mathcal{G}_1,\mathcal{G}_2,\dots,\}$  for the trained model to solve. However, it was performed only once at the beginning of the inference. In this paper, we propose Dynamic Data Augmentation (D2A), an enhancement that generates new augmented instances each time the solver fails to find a better solution (i.e., getting trapped in local optima) within a consecutive maximum of  $T_{\mathrm{D2A}}$  steps. This allows for more diverse searches. Further details regarding training and inference algorithms are provided in Appendix C.
5 Guided infeasible region exploration (GIRE)
As demonstrated in Figure 2, the feasible region is usually fragmented with isolated islands. Masking in neural solvers typically restricts the search to feasible regions, which may lead to inefficient search trajectories or failure to find global optima. Our GIRE explores both feasible and infeasible regions, fostering shortcut discovery, more promising boundary searches, and the identification of possibly isolated regions. We now illustrate our GIRE by handling the capacity constraint in CVRP.
Figure 2: A search space example.
Definition 2 (Search space of CVRP): We define the feasible search space  $\mathcal{F}$  as the set of solutions that satisfy both TSP and the (CVRP-specific) capacity constraint; the infeasible search space  $\mathcal{U}$  as the set of solutions that satisfy the TSP constraint but not the capacity constraint; and the  $\epsilon$ -feasible search space  $\epsilon-\mathcal{F} \subseteq \mathcal{U}$  as the set of solutions with a capacity violation percentage not exceeding  $\epsilon$ .
Feature supplement. GIRE suggests supplementing Violation Indicator (VI) features and Exploration Statistics (ES) features into the policy network to identify specific constraint violations and understand its ongoing exploration behaviour. The VI features flag the specific infeasible portions of the current solution. For CVRP, we use two binary variables indicating if the cumulative demands exceed the capacity before or after visiting a particular node, respectively, which are treated as node features during the node embedding generation. Such an idea could work with most VRP constraints, e.g., indicating the nodes that violate their time windows (TW) for CVRP-TW. The ES features provide the network with ongoing exploration behaviour statistics to guide future exploration behaviour. We define  $\mathcal{H}_t\coloneqq \{(\tau_{t'}\to \tau_{t' + 1})\}_{t' = t - T_{\mathrm{his}}}^{t - 1}$  as the collection of the most recent  $T_{\mathrm{his}}$  steps of solution transition record (if they exist). The ES features  $\mathcal{I}_t$  consist of estimated feasibility transition probabilities derived from  $\mathcal{H}_t$ , including  $P(\tau \in \mathcal{F},\tau '\in \mathcal{U})$ ,  $P(\tau \in \mathcal{U},\tau '\in \mathcal{F})$ ,  $P(\tau \in \mathcal{F},\tau '\in \mathcal{F})$ ,  $P(\tau \in \mathcal{U},\tau '\in \mathcal{U})$ ,  $P(\tau '\in \mathcal{F}|\tau \in \mathcal{U})$ ,  $P(\tau '\in \mathcal{U}|\tau \in \mathcal{F})$ ,  $P(\tau '\in \mathcal{F}|\tau \in \mathcal{F})$ , and  $P(\tau '\in \mathcal{U}|\tau \in \mathcal{U})$ , along with a binary indicator w.r.t the feasibility of the current solution  $\tau_t$ . In order to make the policy network decisions dependent on these ES features, we introduce two hypernetworks [53], namely,  $\mathrm{MLP}_{\mu}$  and  $\mathrm{MLP}_{\lambda}$ , which take ES features  $\mathcal{I}_t$  as inputs and generate the parameters of the last decoder layer, i.e.,  $W_{\mu}^{O}$ ,  $W_{\lambda}^{O}$  of Eq. (3), respectively. We adopt the structure  $(9\times 8\times d)$ , and share the first hidden layer between  $W_{\mu}^{O}$  and  $W_{\lambda}^{O}$  to reduce computational costs.
Reward shaping. Moreover, GIRE employs reward shaping:
$$
r _ {t} ^ {\text {G I R E}} = r _ {t} + \alpha \cdot r _ {t} ^ {\text {r e g}} + \beta \cdot r _ {t} ^ {\text {b o u n s}} \tag {4}
$$
to guide the reinforcement learning, where  $r_t$  is the original reward,  $r_t^{\mathrm{reg}}$  regulates extreme exploration behaviours;  $r_t^{\mathrm{bonus}}$  encourages exploration in the  $\epsilon$ -feasible regions; and  $\alpha$  and  $\beta$  are reward shaping weights (we use 0.05 for both). Here, the regulation  $r_t^{\mathrm{reg}}$  is determined by an entropy measure  $\mathbb{H}[P]$  of the estimated conditional transforming probabilities  $P_t(\mathcal{U}|\mathcal{U}) = P(\tau' \in \mathcal{U} | \tau \in \mathcal{U})$  and  $P_t(\mathcal{F}|\mathcal{F}) = P(\tau' \in \mathcal{F} | \tau \in \mathcal{F})$ :
Figure 3: Plot of  $\mathbb{H}[P]$
$$
r _ {t} ^ {\text {r e g}} = - \mathbb {E} \left[ r _ {t} \right] \times \left[ \mathbb {H} \left[ P _ {t} (\mathcal {U} | \mathcal {U}) + \mathbb {H} \left[ P _ {t} (\mathcal {F} | \mathcal {F}) \right] \right], \right.
$$
$$
\mathbb {H} [ P ] = \operatorname {C l i p} \left\{1 - c _ {1} \log_ {2} \left[ c _ {2} \pi e P (1 - P) \right], 0, 1 \right\}, c _ {1} = 0. 5, c _ {2} = 2. 5, \tag {5}
$$
where expectation  $\mathbb{E}[r_t]$ , that suggests the magnitude of  $r_t^{\mathrm{reg}}$ , is estimated during training; the entropy measure  $\mathbb{H}[P]$ , as shown in Figure 3, imposes larger penalties when  $P$  is either too high or too low (indicating extreme exploration behaviour). The bonus  $r_t^{\mathrm{bonus}}$  utilizes a similar reward function as the regular reward  $r_t$ ; however, it only considers an infeasible but  $\epsilon$ -feasible solution as a potential new best-so-far solution. More illustrations and discussions on GIRE designs are detailed in Appendix D.
We conduct experiments on TSP and CVRP, with sizes  $N = 20$ , 50, 100 following the conventions [9, 13, 37]. Training and test instances are uniformly generated following [3]. For NeuOpt, we use  $K = 4$ ; the initial solutions are sequentially constructed in a random fashion for both training and inference. Results were collected using a machine equipped with NVIDIA 2080TI GPU cards and an Intel E5-2680 CPU at 2.40GHz. More hyper-parameter details, discussions, and additional results are available in Appendix E. Our PyTorch code and pre-trained models are publicly available<sup>2</sup>.
6.1 Comparison studies
Setup. In Table 1, we benchmark our NeuOpt (TSP) and NeuOpt-GIRE (CVRP) against a variety of neural solvers, namely, 1) L2P solvers:  $GCN + BS$  [14] (TSP only),  $Att - GCN + MCTS$  [6] (TSP only),  $GNN + GLS$  [40] (TSP only),  $CVAE - Opt - DE$  [43],  $DPDP$  [42] (state-of-the-art),  $DIMES$  [7] (TSP only),  $DIFUSCO$  [15] (TSP only, state-of-the-art); 2) L2C solvers:  $AM + LCP$  [33],  $POMO$  [4],  $Pointerformer$  [32] (TSP only),  $Sym-NCO$  [13],  $POMO + EAS$  [5],  $POMO + EAS + SGBS$  [34] (state-of-the-art); and 3) L2S solvers: Costa et al. [16] (TSP only), Sui et al. [17] (TSP only), Wu et al. [39], NLNS [8] (CVRP only),  $NCE$  [37] (CVRP only),  $DACT$  [9] (state-of-the-art). To ensure fairness, we test their publicly available pre-trained models on our hardware and test datasets. Those marked with  $\ddagger$  are sourced from their original papers due to difficulties in reproducing, among which we find potential issues marked with  $\#$ . More implementation details are listed in Appendix E. Following the conventions [5, 9, 34], we report the metrics of objective values and optimality gaps averaged on a test dataset with 10k instances, where the total run time is measured under the premise of using one GPU for neural methods and one CPU for traditional ones. The gaps are computed w.r.t. the exact solver Concorde [54] for TSP and the state-of-the-art traditional solver HGS [21] for CVRP. We also include the LKH [20, 51] as baselines. However, we note that it is hard to be absolutely fair when comparing the run time between those CPU-based traditional solvers and GPU-based neural solvers. The baselines are grouped, where the last group comprises variations of our NeuOpt, differentiated by the number of augments (marked as 'D2A=') and the number of inference steps (marked as 'T=').
TSP results. Compared to L2P solvers, NeuOpt (D2A=1, T=1k) surpasses GCN+BS, CVAE-Opt-DE, and GNN+GLS in all problem sizes with shorter run time. With increased T, NeuOpt continues reducing the gaps, and outshines the state-of-the-art DIFUSCO solver with less time at T=10k steps. The NeuOpt (D2A=5, T=1k), with more augmentations, shows lower gaps than NeuOpt (D2A=1, T=5k) in the same solution search count, where it achieves the lowest gap of 0.00% at (D2A=5, T=5k) on all sizes. Despite the longer run time compared to DPDP and Att-GCN+MCTS, their high efficiency is limited to TSP, and our NeuOpt could be potentially boosted by leveraging heatmaps
$^{2}$ https://github.com/yining043/NeuOpt
similar to theirs. As for L2C solvers, NeuOpt (D2A=5, T=3k) beats all baselines on TSP-100 with less time, including the state-of-the-art POMO+EAS+SGBS solver. When pitted against L2S solvers which ours also belongs to, NeuOpt is able to at least halve their gaps using a much shorter run time, demonstrating a substantial improvement. Lastly, our NeuOpt (D2A=5, T=5k) is one of the neural solvers (including DPDP) that can solve TSP-100 to near-optimal (i.e., 0.00%) in a reasonable time.
CVRP results. Most L2P solvers fail to handle CVRP. The remaining ones, CVAE-Opt-DE and DPDP, are significantly outstripped by our solver  $(\mathrm{D}2\mathrm{A} = 5$ $T = 6k)$  with less run time. For L2C solvers, our NeuOpt-GIRE  $(\mathrm{D}2\mathrm{A} = 1$ $T = 10k)$  could exceed the previous state-of-the-art POMO solver. Furthermore, with  $(\mathrm{D}2\mathrm{A} = 5$ $T = 6k)$  and  $(\mathrm{D}2\mathrm{A} = 5$ $T = 40k)$ , our method surpasses POMO+EAS and POMO+EAS+SGBS (long) with less run time, respectively, even though they equip with extra post-hoc per-instance processing boosters. We note that such post-processing strategies (e.g., per-instance active search) could potentially be integrated into ours for further performance gains. Compared to L2S solvers, due to the advances of our GIRE, even NeuOpt-GIRE  $(\mathrm{D}2\mathrm{A} = 1$ $T = 5k)$  could reduce the gaps of those (masking-based) L2S solvers by an order of magnitude in much less time on CVRP-100. Notably, we are the first L2S solver to surpass the strong LKH-3 solver on CVRP.
Table 2: Comparisons of our proposed RDS decoder and GIRE scheme versus existing designs.
Methods	Encoder Type	Decoder Type	Feasibility Scheme	TSP-100	Time↓	CVRP-20
Size(m)	Obj.↓	Gap↓	Size(m)	Obj.↓	Gap↓
DACT (2-opt)	d=64,FF,DAC-Att	DACT	masking	0.281	7.933	1.73%	134s	0.281	6.172	0.14%
DACT-U (2-opt)	d=128,MLP,Synth-Att	DACT	masking	0.633	7.822	0.32%	171s	0.633	6.171	0.13%
DACT-U-GIRE (2-opt)	d=128,MLP,Synth-Att	DACT	GIRE	0.633	7.822	0.32%	171s	0.633	6.168	0.08%
NeuOpt (K=2)	d=128,MLP,Synth-Att	RDS	masking	0.683	7.817	0.24%	122s	0.683	6.180	0.27%
NeuOpt-GIRE (K=2)	d=128,MLP,Synth-Att	RDS	GIRE	0.683	7.817	0.24%	122s	0.685	6.167	0.05%
NeuOpt-GIRE (K=4)	d=128,MLP,Synth-Att	RDS	GIRE	0.683	7.798	0.00%	133s	0.685	6.163	0.00%
Table 3: Effects of GRUs, move stream  $\mu$ , and edge stream  $\lambda$ .
Methods	TSP-100	CVRP-20
Size(M)	Obj.↓	Size(M)	Obj.↓
w/o-GRUs	0.468	7.804	0.470	6.165
w/o-μ	0.617	7.806	0.620	6.165
w/o-λ	0.617	7.799	0.620	6.164
Ours	0.683	7.798	0.685	6.163
Table 4: Effects of dynamic data augmentation (D2A).
Inference Type	TSP-100 Gap↓	CVRP-100 Gap↓
w/o-D2A (T=5k)	0.09%	1.00%
w-D2A (T=5k)	0.05%	0.87%
w/o-D2A (T=10k)	0.04%	0.71%
w-D2A (T=10k)	0.02%	0.60%
Figure 4: Effects of GIRE designs.
Effectiveness analysis. To further reveal the effectiveness of our RDS decoder and GIRE, we conduct a comparison with the state-of-the-art L2S solver, DACT [9]. We upgraded it to DACT-U for fairness, which uses identical embeddings and encoders as our NeuOpt. The evaluation criteria include the model size, the best objective value (averaged over four training runs with  $T = 1k$  inference steps on 1k validation instances), the gaps w.r.t. the best solver, and the run time. As evidenced in Table 2, our GIRE scheme consistently boosts both NeuOpt and DACT-U on CVRP-20, reducing the mask computation time while marginally increasing model size, which reveals the generality of our GIRE. Moreover, when all are used for 2-opt, NeuOpt ( $K = 2$ , for TSP) and NeuOpt-GIRE ( $K = 2$ , for CVRP) surpass DACT-U (for TSP) and DACT-U-GIRE (for CVRP), respectively, with less run time. This is further amplified by increasing  $K$  to 4 for NeuOpt at the cost of slightly increased run time.
6.2 Ablation studies
Ablation on RDS decoder. In Table 3, we remove the key components of our RDS decoder including the GRUs, the move stream  $\mu$ , and edge stream  $\lambda$ , respectively, and report the model size and the best objective value averaged over four training runs. As depicted, all components largely contribute to effective contextual modeling for parameterizing the k-opt decoder on both TSP-100 and CVRP-20.
Ablation on D2A inference. In Table 4, we gather the gaps (similar to Table 1) with and without our D2A design when a single data augmentation is leveraged during inference with  $\mathrm{T} = 5\mathrm{K}$  and  $\mathrm{T} = 10\mathrm{K}$ . The results confirm that our D2A consistently enhances the inference on both TSP-100 and CVRP-100.
Ablation on GIRE designs. We examine scenarios, considering the inclusion or exclusion of Violation Indicator (VI), Exploration Statistics (ES), and Reward Shaping (RS), using eight combinations of them during the training of NeuOpt on CVRP-20. We draw the box plots of the best objective values achieved across eight training runs for each scenario in Figure 4. We can conclude that: 1) the supplement of VI consistently enhances training outcomes, irrespective of the presence of RS; 2) the supplement of ES may bring improvement only when RS is present; and 3) our complete GIRE design  $(\mathrm{VI} + \mathrm{ES} + \mathrm{RS})$  yields the lowest objective values and demonstrates superior stability.
Ablation basis moves. In Figure 5 (a), we depict the training curves of NeuOpt on TSP-100 with and without the learnable S-move. When removing the learnable S-move and opting for
a random selection of the anchor node  $x_{a}$ , a significant degradation in the final performance can be observed, underscoring the necessity of the learnable S-move design. Meanwhile, Figure 5 (b)
(a) Effects of S-move.
(b) Effects of E-move.
Figure 5: Effects of basis moves.
Table 5: Generalization (10 runs) on TSPLIB.
AM -mix	POMO -mix	AMDKD (POMO)	DACT (sol.10k)	Ours (sol.10k)
Avg.↓	Best↓	Avg.↓	Best↓
19.59%	0.92%	1.18%	3.29%	1.59%	0.85%	0.50%
Table 6: Generalization (10 runs) on CVRPLIB.
AM
-mix	POMO
-mix	AMDKD
(POMO)	DACT (sol.10k)	Ours (sol.10k)
Avg.↓	Best↓	Avg.↓	Best↓
15.87%	8.05%	5.77%	5.21%	3.68%	4.80%	3.27%
Table 7: Results on  $N = {200}$  .
Methods	TSP-200	CVRP-200
Gap.↓	Time.↓	Gap.↓	Time.↓
LKH [20, 51]	0.00%	2.3h	1.17%	21.6h
Ours (D2A=5,T=10k)	0.04%	4.7h	0.68%	9.6h
Ours (D2A=5,T=20k)	0.02%	9.4h	0.48%	19.2h
Ours (D2A=5,T=30k)	0.01%	14.1h	0.39%	1.2d
Table 8: Influence of  $K$  .
K	TInf.)↓	T(tr.)↓	Gap↓
2	2m02s	20m	0.30%
3	2m07s	21m	0.15%
4	2m13s	22m	0.05%
5	2m19s	24m	0.01%
6	2m26s	25m	0.00%
Figure 6: Influence of  $\alpha$  and  $\beta$ .
presents the pointplots with confidence intervals that show the performance of our NeuOpt on TSP-100 with and without E-move during decoding across varying preset  $K$ . When E-move is absent (dotted blue line), the model, downgraded to performing fixed  $K$ -opt only, exhibits diminished performance on larger  $K$ . Conversely, our NeuOpt (solid orange line) could further benefit from a larger  $K$ , due to its flexibility in determining and combining different  $k$  across search steps.
6.3 Generalization and scalability studies
In Table 5 and Table 6, we further evaluate the generalization capability of our NeuOpt models on more complex instances from TSPLIB [55] and CVRPLIB [56], respectively. Our NeuOpt achieves lower gaps than DACT [9] and L2C solvers (AM [3] and POMO [4]), and even shows superiority over the AMDKD method [11] that is explicitly designed to boost the generalization of POMO through knowledge distillation. Beyond generalization, our NeuOpt also exhibits notable scalability. As shown in Table 7, when trained directly for size 200, NeuOpt finds close-to-optimal solutions and still surpasses the strong LKH-3 solver [20] on CVRP-200. Note that existing L2S solvers, e.g., DACT [9] may struggle with training on such scales. More details and discussions are available in Appendix E.
6.4 Hyper-parameter studies
Influence of preset  $K$ . In Table 8, we display the performance of NeuOpt on TSP-100, as well as the corresponding inference time (T=1k) and the training time (per epoch) for varying  $K$  values. The results highlight trade-offs between better performance and increased computational costs.
Influence of GIRE hyper-parameters. Figure 6 depicts the influence of  $\alpha$  and  $\beta$  in Eq. (4) on CVRP-20, where we fix one while varying the other, investigating both extremely smaller (0.01) and larger (0.1) values. The results suggest that more effective reward shaping occurs when the weights are moderate. Please refer to Appendix D for discussions on more GIRE hyper-parameters.
7 Conclusions and limitations
In this paper, we introduce NeuOpt, a novel L2S solver for VRPs, that performs flexible k-opt exchanges with a tailored formulation and a designed RDS decoder. We also present GIRE, the first scheme to transcend masking for constraint handling based on feature supplement and reward shaping, enabling autonomous exploration in both feasible and infeasible regions. Moreover, we devise a D2A augmentation method to boost inference diversity. Despite delivering state-of-the-art results, our work still has limitations. While NeuOpt exhibits better scalability than existing L2S solvers, it falls short against some L2P solvers (e.g., [6, 7]) for larger-scale TSPs. Possible solutions in future works include: 1) integrating divide-and-conquer strategies as per [6, 57-60], 2) reducing search space via heatmaps as predicted in [6, 15], 3) adopting more scalable encoders [61, 62], and 4) refactoring our code with highly-optimized CUDA libraries as did in [7, 15]. Besides enhancing scalability, future works can also focus on: 1) applying our GIRE to more VRP constraints and even beyond L2S solvers, 2) integrating our method with post-hoc per-instance processing boosters (e.g., EAS [5]) for better performance, and 3) enhancing the generalization capability of our NeuOpt on instances with different sizes/distributions (e.g., by leveraging the frameworks in [11, 63]).
This research was supported by the Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 1 grant.
[59] Xuanhao Pan, Yan Jin, Yuandong Ding, Mingxiao Feng, Li Zhao, Lei Song, and Jiang Bian. H-TSP: Hierarchically solving the large-scale travelling salesman problem. In AAAI Conference on Artificial Intelligence, 2023.
[60] Sirui Li, Zhongxia Yan, and Cathy Wu. Learning to delegate for large-scale vehicle routing. Advances in Neural Information Processing Systems, 34:26198-26211, 2021.
[61] Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer. In Advances in Neural Information Processing Systems, volume 35, pages 14501-14515, 2022.
[62] Sebastian Jaszczur, Aakanksha Chowdhery, Afroz Mohiuddin, LUKASZ KAISER, Wojciech Gajewski, Henryk Michalewski, and Jonni Kanerva. Sparse is enough in scaling transformers. In Advances in Neural Information Processing Systems, volume 34, pages 9895–9907, 2021.
[63] Jianan Zhou, Yaoxin Wu, Wen Song, Zhiguang Cao, and Jie Zhang. Towards omni-generalizable neural methods for vehicle routing problems. In International Conference on Machine Learning, 2023.
Learning to Search Feasible and Infeasible Regions of Routing Problems with Flexible Neural k-Opt (Appendix)
A Action factorization examples
Figure 7 depicts examples of our factorization method using combinations of basis moves (S-move, I-move, and E-move) to represent different k-opt exchanges. Initiated from a TSP-9 instance (leftmost), we list examples from 1-opt to 4-opt, where the number of I-move corresponds to varying  $k$  values, leading to distinct new solutions (rightmost). This demonstrates the flexibility of our factorization.
Figure 7: Examples of using the basis moves to factorize 1-opt (void action), 2-opt, 3-opt, and 4-opt.
B NeuOpt encoder
Given a state  $s_t = \{\mathcal{G}, \tau_t, \tau_t^{\mathrm{bsf}}\}$ , where  $\mathcal{G}$  is the current instance,  $\tau_t$  is the current solution, and  $\tau_t^{\mathrm{bsf}}$  is the best-so-far solution before step  $t$ , the encoding process first translates the raw features of state into node embeddings. These embeddings are subsequently refined through  $L = 3$  stacked encoders.