## 6 Results and Discussion
### Results on Scenario 2: Increase Forklift Number to \(K=4\)  
In this scenario, we increase the number of forklifts to \(K=4\) and reset the start position of the agent in order to further investigate performance under a complex environment. The four forklifts begin moving at \(t=30\). The results are shown in Figures 13 and 14. As shown in Figure 13, the number of refusals with the RL method is 54, while the number with the proposed FSA-RL method is 2. Compared with the RL method,  
Figure 14: Examples of generated paths of scenario 2: (**a**) classic RL method, path length 84; (**b**) proposed FSA-RL method, path length 78. Figure 13: Simulation results of scenario 2 (increasing forklift number to \(K=4\)): (**a**) refusals, (**b**) average task success rate, (**c**) average reward of generated paths, and (**d**) average length of generated paths. the proposed method reduces the refusals by about 96.3%. Compared with the results in Figure 11, it can be seen that the number of refusals increases. Consequently, the average task success rate of RL is 0.46 and that of FSA-RL is 0.98 (Figure 13b), for an improvement of about 113.0%. As the number of forklifts increases, the refusals increase and the task success rate decreases. For the generated paths, the average reward decreases for both the RL and FSA-RL methods. The average reward of FSA-RL is improved by 63.1% compared to that of the RL method. Figure 13c shows that the average reward for the RL method drops to \(-2.60\), while that of the proposed FSA-RL decreases only slightly to \(-0.96\) in comparison to Figure 11c. Meanwhile, the average length also increases, as shown in Figure 13d. Figure 14 shows examples of the generated paths. Table 6 shows the risk level of each door from time step 36 to 46. In Figure 14a, the agent visits workstations \(o_{2}\to o_{1}\to o_{3}\) to perform collection tasks. Then, it passes through the door #2 and visits \(g_{4}\to g_{2}\to g_{1}\) for delivery tasks. In detail, the agent requests to open door #2 at time step \(t=37\) and enters the corridor at time step \(t=38\). The risk values of door #2 are \(r(B_{2},38)=0.147\) and \(r(B_{2},39)=0.111\). Thus, the agent encounters a greater risk of being refused. The refusal results in Figure 13a also verify this point, with the agent being refused by the door 54 times. Figure 14b shows the generated path. The visit order for collection tasks is the same as with the RL method, \(o_{2}\to o_{1}\to o_{3}\). After finishing the collection tasks, the agent moves to the other workshop through door #1. The agent enters the corridor at \(t=44\), where the risk value is \(r(B_{1},44)=0.006\). Thus, the agent is only refused twice in 100 simulation runs. The simulation results show that the proposed RL-based double-layer controller is efficient in finding an optimal path for a mobile robot working in human-shared environments. The number of refusals is greatly reduced compared to the classic RL method. The quality of the generated path is improved from the viewpoint of both task success rate and path reward. Simulation results demonstrate that the proposed method outperforms traditional RL approaches. One advantage comes from our algorithm's ability to decompose the complex path planning problem into several smaller subproblems with reduced search spaces, making them easier to solve. In traditional RL approaches, the size of the search space is \(|S|\times 2^{2I}\) due to the need to handle all tasks and states. In contrast, our hierarchical method significantly reduces the complexity; the high-level planner operates over a smaller abstract state space of size \(|Q|\), while the low-level planner operates over \(|S|\). For example, the size of the search space for the RL algorithm is \(|S|\times 2^{2I}=20\times 22\times 2^{6}=28160\). In our proposed algorithm, the size of high-level search space is \(|Q|=26\), while the size of low-level search space is \(|S|=22\times 20=440\). Each subproblem needs to be learned only once, eliminating the need for repeated learning.