# Multi-Scenario Robust Distributed Permutation Flow Shop Scheduling Based on DDQN
## 5 Experiments
### DDQN Performance Evaluation for MSRDPFSP  
Since there is no publicly available dataset for testing MSRDPFSP, the publicly available dataset of DPFSP is still used. The dataset mainly tests the data of 500 workpieces processed by 20 machine tools allocated to different workshops. The convergence curves of the random search strategy algorithm and the DDQN algorithm with 2000 iterations are compared. The test results are shown in Figure 25 below.  
From the convergence curve, it can be seen that after the introduction of the DDQN algorithm to learn the strategy, it shows faster convergence speed and better results compared to the random selection strategy during the iteration process. Specifically, once the experience pool has accumulated enough experience and gradually starts to select from randomly selected actions to high q-value actions, the convergence speed of DDQN starts to be significantly ahead of the randomly selected algorithm. This empirical superiority underscores DDQN's capacity to effectively guide the scheduling policy to converge towards the optimization goal and highlights its enhanced learning ability and optimization performance for the MSRDPFSP with uncertain processing times.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameters** & **Hidden Meaning** & **Parameter Value** \\ \hline lr & Discount rate & 0.005 \\ batch\_size & Number of training samples & 32 \\ EPSILON & Initial greedy rate & 1 \\ EPSILON\_decay & Decay rate & 0.998 \\ EPSILON\_min & Minimum greedy rate & 0.1 \\ GAMMA & Discount rate & 0.9 \\ TARGET\_REPLACE\_ITER & Synchronization frequency & 16 \\ MEMORY\_CAPACITY & Experience pool size & 128 \\ \hline \hline \end{tabular}
\end{table}
Table 4: DDQN parameter setting.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \(\mathbf{N\times M\times F}\) & **DDQN** & **NEH** & \(\mathbf{N\times M\times F}\) & **DDQN** & **NEH** \\ \hline \(100\times 10\times 2\) & 0.72 & 5.15 & \(200\times 20\times 2\) & 0.29 & 5.28 \\ \(100\times 10\times 3\) & 1.93 & 6.87 & \(200\times 20\times 3\) & 0.08 & 2.83 \\ \(100\times 10\times 4\)  
Taking the data of 500-20-7 as an example, the initial solution of NEH for the DPFSP in MSRDPFSP each workshop is initially not balanced. Workshop 1 has a faster machining speed because of the use of conveyor belts in the workshop. Over 2000 iterations, the stochastic optimization strategy distributes scheduling actions nearly uniformly (approximately 500 executions per strategy), whereas the DDQN algorithm selectively executes the insertion strategy 800 times after learning its higher reward potential. Specifically, DDQN prioritizes inserting workpieces from the longest-processing-time workshop into the shortest-processing-time one. This learning does not take too much time: the time taken for executing the DDQN strategy for the 2000 th time is 1025 s, while that of the stochastic strategy is 1000 s, as shown in Figure 26 below.  
Plant Simulation has a built-in genetic algorithm component for the workshop scheduling problem, which is compared with the hybrid algorithm proposed in this paper, using the same data, with the initial population size set to 20, 100 iterations, and a final elapsed time of 1000 s. The results obtained are shown in Figure 27 below. As illustrated in the figure, the experimental comparison demonstrates that the approach proposed in this paper outperforms the hybrid algorithm, which in turn significantly surpasses this genetic algorithm module.  
Figure 25: Comparison of 500 workpieces on 20 machines across different workshops.