# Graph-Driven Deep Reinforcement Learning for Vehicle Routing Problems with Pickup and Delivery
## 6 Conclusions  
In this study, we concentrate on two variants of pickup and delivery problems, i.e., the conventional VRP-PD and the improved VRP-PD, which have broad applications in logistics, robotics, and so on. We propose a novel deep reinforcement learning approach, which engages an encoder-decoder framework, to tackle them. The encoder utilizes stacked GCMs to learn hierarchical graph features and generate informative tokens of nodes and edges. Afterwards, the decoder uses an attention-driven compatibility layer with lightweight computation to output a probability distribution for the next node selection. Extensive experiments on synthetic and real-world benchmark datasets demonstrate that the proposed GDRL outperforms a variety of heuristic and DRL-based algorithms on problems with different scales and node distributions. Despite its demonstrated effectiveness, the proposed approach presents two notable limitations. First, the GDRL encoder relies on stacked graph convolutional layers to capture node and edge information. While effective, this architecture demands substantial computational resources, resulting in suboptimal runtimes for large-scale instances involving more than 500 customer nodes. Second, the current framework is specifically designed for VRP-PD scenarios and does not readily extend to more complex variants, such as those involving service time windows. Future efforts will focus on generalizing the approach to address more realistic and operationally relevant routing problems, including vehicle routing with time windows and electric vehicle routing with charging constraints. Conceptualization, D.Y. and Q.G. ; methodology, Q.G. ; software, B.O. ; validation, D.Y., Q.G., and B.O. ; formal analysis, B.Y. ; investigation, B.O. ; resources, H.C.; data curation, B.O. ; writing--original draft preparation, Q.G. ; writing--review and editing, D.Y. ; visualization, B.Y. ; supervision, H.C.; project administration, H.C.; funding acquisition, D.Y. All authors have read and agreed to the published version of the manuscript. This work was supported in part by the National Natural Science Foundation of China under Grant No. 62306232, the Natural Science Basic Research Program of Shaanxi Province under Grant No. 2023-JC-QN-0662, and the State Key Laboratory of Electrical Insulation and Power Equipment under Grant No. EIPE23416. **Institutional Review Board Statement:** Not applicable. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Sampling Size**} & \multicolumn{3}{c}{**320**} & \multicolumn{3}{c}{**640**} & \multicolumn{3}{c}{**1280**} & \multicolumn{3}{c}{**2560**} \\ \cline{2-13}  & **Length** & **Gap** & **Time (s)** & **Length** & **Gap** & **Time (s)** & **Length** & **Gap** & **Time (s)** & **Length** & **Gap** & **Time (s)** \\ \hline AM & 53.69 & 200.28\% & 1072.76 & 52.98 & 198.82\% & 2000.35 & 51.89 & 194.16\% & 3600 & 49.85 & 182.76\% & 3600 \\ POMO & 18.46 & 3.24\% & 44.02 & 18.29 & 3.16\% & 185.43 & 18.16 & 2.95\% & 811.15 & 18.03 & 2.27\% & 3600 \\ GDRL & 17.88 & 0.00\% & 57.15 & 17.73 & 0.00\% & 211.79 & 17.64 & 0.00\% & 1050.38 & 17.63 & 0.00\% & 3600 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Routing results of different sampling sizes on conventional VRP-PDs. \begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Sampling Size**} & \multicolumn{3}{c}{**320**} & \multicolumn{3}{c}{**640**} & \multicolumn{3}{c}{**1280**} & \multicolumn{3}{c}{**2560**} \\ \cline{2-13}  & **Length** & **Gap** & **Time (s)** & **Length** & **Gap** & **Time (s)** & **Length** & **Gap** & **Time (s)** & **Length** & **Gap** & **Time (s)** \\ \hline AM & 49.56 & 442.83\% & 25.67 & 49.12 & 445.17\% & 51.28 & 48.72 & 446.80\% & 102.94 & 48.19 & 445.14\% & 244.38 \\ POMO & 9.36 & 2.52\% & 0.56 & 9.28 & 3.00\% & 0.91 & 9.19 & 3.14\% & 1.46 & 9.12 & 3.17\% & 2.53 \\ GDRL & 9.13 & 0.00\% & 0.72 & 9.01 & 0.00\% & 1.33 & 8.91 & 0.00\% & 2.05 & 8.84 & 0.00\% & 3.95 \\ \hline \hline \end{tabular}
\end{table}