## 4 Methodology
### Sequential-Placement First Heuristic (\(\mathit{SPFH}\))
11:while An item in the \(top_{2}[W]\) is not empty do
12:for all\(q_{1}\in Q_{1}\)do
13:for all\(t_{1}\in top_{1}\)do
14:\(l_{1}\leftarrow\)Finding the position index of \(t_{1}\) in \(q_{1}\)
15:for all\(t_{2}\in top_{2}\)do
16:for all\(i\in[l_{1},len(q_{1})]\)do
17:\(q_{2}\leftarrow\)Insert \(t_{2}\) sequentially into \(q_{1}\) at \(i\)th item, and generate a new sequence. 18:\(Q_{2}\gets Q_{2}\cup\{q_{2}\}\)
19:\(Q_{1}\leftarrow\varnothing\), \(Q_{1}\gets Q_{2}\), \(Q_{2}\leftarrow\varnothing\), \(top_{1}\gets top_{2}\)
20:\(top_{2}[W]\leftarrow\)Get the top target container in each stack of \(W\) and remove them from corresponding stack. 21:return\(Q_{1}\)
```  
**Algorithm 2** The procedures: AutoRetrieval and GenerateSequence  
The detailed simulation and evaluation procedure consists of the following steps. Firstly, we apply the LL rule to identify the optimal stack \(s^{*}\), which will minimize the addition of ENBC among all assignable slots to which we can move the blocking container directly. However, the earlier reallocations of the blocking containers occupy reasonable placements and may lead subsequent blocking containers to lose the well-stacking slots. Hence, we will consider the impact of current reallocation decisions on subsequent blocking containers and determine whether there are better solutions than the current selection. Suppose the selected stack \(s^{*}\) is a sequential-stack for the current blocking container. In this case, we will check for some inverted-placement containers that satisfy the MSS rule to increase the chances of choosing a well-stacking position for other blocking containers. Similarly, suppose the selected stack \(s^{*}\) is a inverted-stack, we will try to find a stack \(s^{\prime}\) satisfying the FSS rule that the stack \(s^{\prime}\) is a sequential-stack for the current blocking container after moving the topmost container in stack \(s^{\prime}\) to another stack. The optimal result is that the non-forced relocating container obtains a new well-stacking placement. Otherwise, relocating the non-forced container will increase the final operational cost in the future. Moreover, when stack \(s^{*}\) provides a level-placement for the current relocating container, neither the MSS rule nor the FSS rule is applicable because they may exacerbate the problem. Algorithm 3 presents the pseudo-code for the inner level of \(SPFH\). ```
1:procedureSimulation-Evaluation(\(q\), \(S\), \(sol\))
2:\(P\gets q\),\(S_{cur}\gets S\);
3:while\(P\) is not empty do
4:\(p\leftarrow\) obtains the first item of \(P\)
5:\(B_{p}\leftarrow\) obtain the blocking containers of the target container \(p\)
6:for all\(b\in B_{p}\)do
7:\(opr\leftarrow\varnothing\), \(s^{*}\gets LL(b,S_{cur})\)\(\triangleright\) Get the stack for container \(b\) according to LL
8:while\(f(b,w(b),s^{*})=0\)then
9:\(s^{\prime}\gets MSS(s^{*},b,S_{cur})\)\(\triangleright\) Find a stack in satisfying MSS rule
10:if\(s^{\prime}\) is not NULL then
11:\(opr\gets opr+\langle top(s^{\prime}),s^{\prime},s^{*}\rangle\), \(opr\gets opr+\langle b,w(b),s^{*}\rangle\)
12:elseif\(f(b,w(b),s^{*})>0\)then
13:\(s^{\prime}\gets FSS(s^{*},b,S_{cur})\)\(\triangleright\) Find a stack in satisfying FSS rule
14:if\(s^{\prime}\) is not NULL then
15:\(s^{\prime\prime}\gets LL(top(s^{\prime}),S_{cur})\), \(opr\gets opr+\langle top(s^{\prime}),s^{\prime},s^{\prime\prime}\rangle\), \(opr\gets opr+\langle b,w(b),s^{\prime}\rangle\)
16:else
17:\(opr\gets opr+\langle b,w(b),s^{*}\rangle\)
18:else
19:\(opr\gets opr+\langle b,w(b),s^{*}\rangle\)
20:\(S_{cur}\gets S_{cur}\oplus opr\), \(sol\gets sol+opr\)\(\triangleright\) is an operator that indicates that \(opr\) will be executed based on the bay layout \(S_{cur}\) and generate a new layout. 21:AutoRetrieval(\(S_{cur}\),\(sol\),\(P\))
22:return\(\|opr\|\)
```  
**Algorithm 3** Inner level: Policy iteration procedure