# Multi-Scenario Robust Distributed Permutation Flow Shop Scheduling Based on DDQN
## 4 Algorithm Design  
### Search Strategy  
The quality of the initial solution plays a crucial role in the training performance of reinforcement learning algorithms. In the DDQN algorithm, the intelligent body continuously optimizes its strategy by interacting with the environment, and the quality of the initial strategy directly affects the convergence speed of the learning process and the final scheduling performance; if the initial solution is of poor quality, the intelligent body needs a longer training time to explore a high-quality scheduling strategy, resulting in poor strategy convergence performance [42]. This paper addresses the MSRDPFSP by using the excellent initial solution generated based on the NEH algorithm as the policy starting point of the DDQN intelligences and performing continuous iterative learning in a multi-scenario, dynamic uncertainty environment. Although there are some differences between DPFSP and MSRDPFSP in terms of problem complexity and scenario diversity, the experimental results show that the initial solution generated by NEH can significantly outperform the completely random initialization in the robust optimization process. For this initial solution, the following four global search strategies are designed:  
1. Insertion within workshop: It is executed for all workshops. A set of consecutive sequences within a workshop are randomly selected and inserted into a random position in that workshop. If there is optimization, the optimized sequences are returned; otherwise, the pre-optimized sequences are returned, as shown in Figure 15 below.
2. Intra-workshop swapping: It is executed for all workshops. A set of consecutive sequences within a workshop are randomly selected to recombine their positions, and the optimized sequence is returned if there is an optimization; otherwise, the pre-optimized sequence is returned, as shown in Figure 16 below.
3. Swap between different workshops: All workshops are sorted in terms of elapsed time. Specifically, a set of consecutive sequences in the workshop with the longest elapsed time are chosen to be swapped with the same number of consecutive random sequences in another random workshop. If there is an optimization, the optimized sequences are returned; otherwise, the pre-optimized sequences are returned, as shown in Figure 17 below.
4. Insertion between different workshops: All workshops are sorted in terms of elapsed time. Multiple tasks within the workshop taking the longest time are selected and inserted into other workshops with a short elapsed time. If there is an optimization, the optimized sequence is returned; otherwise, the pre-optimized sequence is returned, as shown in Figure 18 below.  
In this paper, four local search strategies are set up for the insertion and exchange within the workshop. Different workshops do not affect each other. Compared with the traditional genetic algorithm, this approach only needs to interact with the simulation model once to make an optimization attempt for all workshops, as long as there is an improvement within the workshop. Even if there is no improvement in other workshops, the update sequence of the improved workshop is retained, while unimproved workshops  
method. This can quickly equalize the load of each workshop, and the neural network can extract more effective information.