## 4 Solving FJSPLSP-LTCC Using DRL  
As described in Section 3.1, the FJSPLSP-LTCC problem is a complex scheduling problem characterized by sequential decision making, involving several interdependent subtasks: operation assignment, sequencing, AGV scheduling, and charging decision making. DRL is an effective tool for solving such problems.  
To mitigate the inherent combinatorial explosion risk in this multi-stage scheduling problem, this study adopts a structured decision-making process by partitioning the overall action space into staged and hierarchical local subspaces. Specifically, machine allocation, operation sequencing, AGV assignment, and charging decisions are conducted separately  
\begin{table}
\begin{tabular}{l l} \hline
**Notation** & **Description** \\ \hline Indices & \\ \(i\) & Index of job, \(i\in J,J=\{1,2,\ldots,J\}\) \\ \(k\) & Index of operation, \(k\in O_{i},O_{i}=\{1,2,\ldots,K\}\) \\ \(m\) & Index of machine, \(m\in M,M=\{1,2,\ldots,M\}\) \\ \(a\) & Index of AGV, \(a\in A\), \(A=\{1,2,\ldots,A\}\) \\ \(c\) & Index of Charger, \(c\in C,C=\{1,2,\ldots,C\}\) \\ \hline \multicolumn{3}{l}{state variables} \\ \(C_{i}\) & The completion time of \(J_{i}\) \\ \(C_{i,k}\) & The completion time of the operation \(k\) of job \(i\) \\ \(C_{max}\) & The maximum completion time of all jobs \\ \(O_{i}\) & Number of operations for the ith job, \(i\in J,J=\{1,2,\ldots\}\) \\ \(M_{i,k}\) & The set of machines available for operation k of job i \\ \(p_{i,k,m}\) & The processing time of operation \(k\) on machine m \\ \(s_{a}\) & The power of AGV \(a\) \\ \(v_{a}\) & The current speed of AGV \(a\) \\ \(\phi_{a}\) & The speed of AGV \(a\) after charging \\ \(T_{\text{mantain}}\) & The time spent on mandatory maintenance battery change \\ \(T_{\text{charge},a}\) & The time spent by AGV \(a\) from start to finish of charging \\ \(T_{\text{charge}1}\) & The time to Opportunity charge \\ \(T_{\text{charge}2}\) & The time to fully charge \\ \(T_{\text{wait}}\) & The time spent waiting in queue for charging \\ \(D_{(a,c)}\) & The distance from AGV \(a\) to charger \(c\) \\ \(D_{(\text{source},\text{dest})}\) & The distance from the current machine of the job to the specified machine \\ \(D_{(c,\text{source})}\) & The distance from the charger \(c\) to the current machine of the job \\ \(s_{a}^{\text{new}}\) & The power after the AGVs perform an action \\ \(P_{\text{charge}}\) & The charging power of AGV \\ \(P_{\text{empty}}\) & The empty transport power of AGV \\ \(P_{\text{load}}\) & The load power of AGV \\ \(T_{\text{transport},a}^{\text{empty}}\) & The empty transport time of AGV \\ \(T_{\text{transport},a}^{\text{load}}\) & The load transport time of AGV \\ \(B_{a_{1}}\) & The start charging time of AGV \(a_{1}\) \\ \(C_{a_{1}}\) & The time required for AGV \(a_{1}\) charging \\ \(y_{a_{1},a_{2}}\) & 1 if AGV \(a_{1}\) is charged before \(a_{2}\), and 0 otherwise \\ M & A large enough positive number \\ \hline \multicolumn{3}{l}{decision variables} \\ \(x_{i,k,m}\) & 1 if machine m processes the operation \(k\) of job \(i\), and 0 otherwise \\ \(y_{i,k,m}\) & 1 if operation \(k\) is processed on machine \(m\), and 0 otherwise \\ \(w_{i,k,a}\) & 1 if the operation \(k\) of job \(i\) is transported by AGVa, and 0 otherwise \\ \(z_{i,k,a}\) & 1 if the AGV \(a\) transports the operation \(k\) of job \(i\), and 0 otherwise \\ \(d_{a}\) & Charging decisions for AGV \(a\), \(d_{a}=\{1,2,3\}\) \\ \(m_{a}\) & Mandatory maintenance battery change decision for AGVa \\ \(y_{a_{1},a_{2}}\) & 1 if AGV \(a_{1}\) is charged before AGV \(a_{2}\), and 0 otherwise \\ \hline \end{tabular}
\end{table}
Table 2: Notations used in FJSPLSP-LTCC.  
within their respective stages, significantly reducing the effective action space at each stage. Additionally, within the PPO algorithm framework, a candidate action sampling strategy is designed to eliminate infeasible or evidently inferior action combinations, thereby enhancing learning efficiency and solution quality.