## 6 Results and Discussion
### Results on Scenario 1: Workstation Number \(I=3\) and Forklift Number \(K=2\)  
Scenario 1 verifies the proposed algorithm under the configuration as shown in Figure 8a. We first construct FSA \(\mathcal{A}_{\varphi}\) from the scLTL formula \(\varphi\) (Equation (4)) with \(I=3\). Figure 9 demonstrates the state transitions of \(\mathcal{A}_{\varphi}\). Based on FSA \(\mathcal{A}_{\varphi}\), we construct the state space of the high-level learner. As shown in Figure 9, it contains 27 nodes, where \(q_{0}=1\) is the initial state and \(q_{F}=0\) is the accept state. From \(q_{0}=1\), the agent randomly chooses an atomic proposition from \(\{pick_{1},pick_{2},pick_{3}\}\) to explore the state space of the high-level learner. Here, we take \(pick_{1}\) as an example. The algorithm maps \(pick_{1}\) to its task position with Equation (9), where \(s_{l}=o_{1}=f(pick_{1})\) and \(\tilde{s}_{l}=s_{0}=f(null_{i})\). Then, the agent sends \((s_{0},o_{1})\) to the low-level learner and evokes it to  
Figure 8: Environment Setting. (**a**) The colored circles show \(I=3\) workstations, \(o_{1},o_{2},o_{3}\). There are six doors, denoted \(\#1,\ldots,\#6\). The colored triangles show the destinations, \(g_{1},g_{2},g_{3}\). There are \(K=2\) forklifts in the environment; the red arrows show their directions. The gray squares are obstacles, while the pink squares show the barrier area of each door. (**b**) The number of forklit s is increased to \(K=4\) and the robot’s start position is reset. Figure 9: State transitions of \(\mathcal{A}_{\varphi}\) converted from scLTL formula \(\varphi\) with \(I=3\). Note that the state transition \(q\leftarrow\delta(q,null_{i})\) exists \(\forall q\in Q\) (not drawn in the figure). obtain \(L(s_{l})\) and the corresponding local path \(x\). After \(L(s_{l})=True\), that is, when \(x\) satisfies the requirement, the agent moves to state \(6\), written as \(\delta_{h}(s_{h}=1,pick_{1},L(s_{l})=True)\rightarrow\dot{s_{h}^{{}^{\prime}}}=6\). Otherwise, if \(L(s_{l})=False\), the generated path \(x\) does not satisfy the requirement and \(\delta_{h}(s_{h}=1,pick_{1},L(s_{l})=False)\to s_{h}=1\). The reward of the learning process is shown in Figure 10. The blue curve shows the learning curve of the classic RL method, which is characterized by substantial oscillatory behavior. The green curve shows the learning curve of the proposed FSA-RL; its reward converges at around \(-1.0\), much larger than classic RL method, which converges at around \(-5.0\). Figure 11 shows the details of the simulation results from the perspective of refused times, average task success rate, average reward, and average length of the generated paths. Note that \"refused times\" refers to the number of times the robot fails to pass through a door along the generated path due to the door rejecting the open request. As shown in Figure 11a, when using the classic RL method in 100 simulation runs, the agent was refused 37 times by the door system and failed to pass the door, achieving an average task success rate 0.63 in Figure 11b. Meanwhile, the proposed method enabled the agent to pass through the door and successfully deliver all workpieces in 100 simulation runs; thus, the number of refusals in Figure 11a is 0, and the average task success rate in Figure 11b is 1.0. Figure 11c,d shows the average reward and average length of the generated paths. There is little difference in the average length between the RL and the proposed FSA-RL methods, with the average length being 79 for RL and 80 for the proposed method. The difference lies in the average reward; as Figure 11c shows, the average reward of the proposed FSA-RL method is \(-0.92\), much larger than \(-1.94\) with the RL method. The reason for this is that the door system refuses to allow the agent to pass through when the agent follows the path generated using RL. Figure 12 shows an example of the generated paths. Figure 12a shows the path generated with RL.