# A Deep Reinforcement Learning-Based Decision-Making Approach for Routing Problems
## 5 Conclusions  
In this study, we present a novel deep reinforcement learning-based decision-making framework (DRL-DM) to address routing problems across varying problem sizes and node distributions. To enhance representation learning, the encoding phase incorporates a batch normalization fronting mechanism and a gate-like threshold block to generate more informative node embeddings. In the decoding phase, we introduce a dynamic-aware context embedding to effectively capture state transitions and graph variations. Additionally, our training strategy leverages multiple starting nodes per instance, enabling broader exploration of the search space through multiple trajectories. Experimental results on two classes of routing problems demonstrate that DRL-DM consistently outperforms heuristic and learning-based baselines while exhibiting superior generalization capabilities. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multicolumn{2}{c}{**Components of Our DRL-DM**} & \multicolumn{3}{c}{**CVRP20**} & \multicolumn{3}{c}{**CVRP50**} & \multicolumn{3}{c}{**CVRP100**} \\ \hline
**BNF** & **GT** & **Context** & **Length** & **Gap** & **Time** & **Length** & **Gap** & **Time** & **Length** & **Gap** & **Time** \\ \hline  & & & 6.41 & 5.08\% & 3 s & 10.99 & 5.98\% & 8 s & 16.82 & 7.48\% & 23 s \\ \hline ✓ & & & 6.37 & 4.43\% & 5 s & 10.91 & 5.21\% & 18 s & 16.66 & 6.45\% & 54 s \\  & ✓ & & 6.31 & 3.44\% & 7 s & 10.80 & 4.15\% & 26 s & 16.46 & 5.18\% & 1 min \\  & & ✓ & 6.27 & 2.79\% & 8 s & 10.72 & 3.38\% & 31 s & 16.30 & 4.15\% & 2 min \\ \hline ✓ & ✓ & & 6.26 & 2.62\% & 9 s & 10.70 & 3.18\% & 37 s & 16.28 & 4.03\% & 2 min \\ ✓ & & ✓ & 6.23 & 2.13\% & 10 s & 10.63 & 2.51\% & 42 s & 16.14 & 3.13\% & 2 min \\  & ✓ & ✓ & & 6.17 & 1.15\% & 12 s & 10.52 & 1.45\% & 50 s & 15.93 & 1.79\% & 3 min \\ \hline ✓ & ✓ & ✓ & & 6.13 & 0.49\% & 14 s & 10.44 & 0.68\% & 1 min & 15.78 & 0.83\% & 3 min \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Ablation study of different components of our DRL-DM on CVRPs. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Value**} & \multicolumn{2}{c}{**TSP20**} & \multicolumn{2}{c}{**TSP50**} & \multicolumn{2}{c}{**TSP100**} & \multicolumn{2}{c}{**CVRP20**} & \multicolumn{2}{c}{**CVRP50**} & \multicolumn{2}{c}{**CVRP100**} \\ \cline{2-13}  & **Length** & **Time** & **Length** & **Time** & **Length** & **Time** & **Length** & **Time** & **Length** & **Time** \\ \hline \(\Psi=1\) & 3.86 & 4 s & 5.74 & 15 s & 7.85 & 42 s & 6.17 & 11 s & 10.51 & 1 min & 15.89 & 2 min \\ \(\Psi=2\) & 3.85 & 5 s & 5.72 & 1  
Notably, recent advancements, like graph neural network (GNN) based approaches [38] and hybrid heuristic-DRL methods [39] are not contained in this study. In GNN-based approaches, the routing environment is represented as a graph where nodes correspond to customers or locations, and edges reflect the connectivity or distance between them. Through iterative message-passing mechanisms, GNNs can capture complex spatial dependencies and relational patterns among nodes, enabling more informed decision-making. Moreover, hybrid methods that combine heuristic strategies with DRL have gained traction. These methods typically employ DRL to learn high-level policies while integrating domain-specific heuristics for local search or post-processing. For example, hybrid heuristic-DRL approaches may use a learned policy to generate an initial solution, which is then refined through traditional heuristics such as 2-opt or simulated annealing. This integration capitalizes on the global learning capacity of DRL while retaining the efficiency and effectiveness of heuristics in local optimization. In addition, we recognize that the deployment of DRL in large-scale routing also carries potential societal and technical risks that warrant careful consideration. First, the training of DRL-DM requires substantial computational resources and prolonged GPU utilization, which can translate into significant energy consumption and associated carbon emissions.