## 5 Experiments
### Performance Comparison of CRGPPO-TKL
#### 5.3.3 Comparison with RN and Action Analysis  
As shown in Table 8, CRGPPO-TKL significantly outperforms the random version (RN) in both average completion time and standard deviation. For example, in the MK08 instance, RN records a mean completion time of 2951.2 with a standard deviation of 1315.7, whereas CRGPPO-TKL achieves a mean of 653.6 and a standard deviation of only 14--demonstrating a substantial performance gap. These results confirm that the proposed method does not rely on random action selection but rather learns to make near-optimal decisions at each step, thereby reducing overall system completion time. To further validate the policy learning capability of CRGPPO-TKL, a detailed action-selection analysis is conducted on MK08, the most challenging instance in terms of average makespan. We conduct 20 independent runs on this instance and record all action selections at each decision point throughout the scheduling process. Actions are then classified and statistically analyzed. Table 10 presents frequency statistics based on charging action types, Table 11 shows results grouped by job-machine combinations, and Figure 10 displays actions that occurred more than 20 times across experiments. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & & **RN** & & & **DQN** & & & **PPO** & & & **CRGPPO-TKL** \\  & **Mean** & **Std** & **ODR** & **Mean** & **Std** & **ODR** & **Mean** & **Std** & **ODR** & **Mean** & **Std** & **ODR** \\ \hline MK01 & 428.1 & 234.1 & 332.1\% & 101.0 & 2.7 & 1.9\% & 101.3 & 24.0 & 2.2\% & **99.1** & 16.6 & **0.0\%** \\ MK02 & 516.2 & 286.2 & 452.2\% & 99.8 & 4.6 & 6.7\% & 93.5 & 20.5 & 0.05\% & **93.5** & 10.7 & **0.0\%** \\ MK03 & 1674.2 & 898.7 & 544.5\% & 292.4 & 16.4 & 12.6\% & 268.4 & 24.8 & 3.3\% & **259.8** & 13.7 & **0.0\%** \\ MK04 & 779.7 & 430.8 & 491.5\% & 142.8 & 8.9 & 8.3\% & 141.7 & 22.8 & 7.5\% & **131.8** & 1.7 & **0.0\%** \\ MK05 & 986.3 & 429.7 & 397.3\% & 206.6 & 2.9 & 4.2\% & 198.4 & 2.6 & 0.02\% & **198.3** & 5.3 & **0.0\%** \\ MK06 & 1570.6 & 857.8 & 717.6\% & 222.1 & 9.6 & 15.6\% & 192.1 & 7.1 & 0.00\% & 195.6 & 6.7 & 1.8\% \\ MK07 & 870.0 & 463.0 & 298.0\% & 231.0 & 5.7 & 5.6\% & 222.3 & 35.7 & 1.67\% & **218.6** & 28.8 & **0.0\%** \\ MK08 & 2951.2 & 1315.7 & 351.5\% & 695.5 & 18.4 & 6.4\% & 758.2 & 307.2 & 16.00\% & **653.6** & 14.0 & **0.0\%** \\ MK09 & 2790.6 & 1327.8 & 405.8\% & 601.5 & 15.5 & 9.0\% & 861.0 & 849.6 & 56.05\% & **551.8** & 6.7 & **0.0\%** \\ MK10 & 3163.2 & 1579.5 & 725.3\% & 427.5 & 16.3 & 11.5\% & 454.6 & 112.6 & 18.60\% & **383.3** & 14.7 & **0.0\%** \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Comparison results of other DRLs and RN. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**DQN**} & \multicolumn{2}{c}{**PPO**} & \multicolumn{2}{c}{**CRGPPO-TKL**} \\  & **Mean** & **Std** & **ODR** & **Mean** & **Std** & **ODR** & **Mean** & **Std** & **ODR** \\ \hline Layout1 & 159.6 & 4.0 & 4.7\% & 153.9 & 20.7 & 1.0\% & 152.4 & 15.3 & 0.0\% \\ Layout2 & 433.1 & 14.8 & 8.5\% & 507.3 & 301.1 & 27.1\% & 399.2 & 9.0 & 0.0\% \\ Layout3 & 324.8 & 13.0 & 12.2\% & 323.3 & 59.8 & 11.7\% & 289.5 & 10.7 & 0.0\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Comparison results of DRL methods under different layouts. Figure 9: Comparison of the ODR mean-variance of other DRL. As shown in Table 10, the agent strongly favors the NC action. OC occurs more frequently than FC, while FC actions are rarely chosen. This strategy reflects real-world operational logic: once the optimal number of AGVs is fixed, each vehicle typically performs continuous transport tasks. When battery levels drop, the agent prefers OC during idle gaps, which minimally disrupts production. In contrast, FC often incurs longer delays and is only selected when necessary. Table 11 indicates that the majority of job-machine combination decisions fall into the MOR + EAM and MOR + LPT categories, accounting for 55.29% and 42.44%, respectively.