## 4 Proposed Algorithm
### Local Search Strategy
#### 4.5.2 Output Designs  
The reinforcement learning results are used to improve the optimization performance of the evolutionary algorithm. Since the training of DDQN neural networks needs to rely on the replay buffer, and the solution sets obtained between adjacent iterations of the algorithm have a certain degree of approximation, this leads to the difficulty of having significant differences in the replay buffers of adjacent iterations, thus making it difficult for neural networks to perform effective parameter optimization. In order to improve the effectiveness of neural network optimization, this paper sets the neural network to be trained at a specific number of iterations. For the evolutionary algorithm, the greater the number of iterations, the better the results and the more iterations are needed for further optimization of the iteration results, so the neural network training is set at 50, 100, and 200 generations. The trained network obtains 40 complete paths by action selection, which are recorded in the set \(G\prime\), to participate in nondominated sorting in the next iteration. The pseudo-code is as follows (Algorithm 3):  
Figure 7: The structure of the DDQN. ```
Input: \(Q(s,a;\ \theta):\) the trained network; epsilon: parameter of greedy policy; N: number of routes; \(V:\) set of workstations; \(\mathit{cunsum}:\) number of workstations; \(s_{0}:\) the outset While\(N\leq 40\)do  Initialize parameters : \(s\leftarrow\ s_{0}\); action \(\leftarrow\varnothing\);  While\(size(action)\leq\mathit{cunsum}\)do  The feasible solutions selected from the unassigned workstations are recorded in \(V_{a}\) if\(V_{a}\



\[\gamma=iter-DDQN_{iter};\ \left\{DDQN_{iter}\in\mathcal{U}\middle|\frac{iter}{2} \leq DDQN_{iter}<iter\right\} \tag{51}\]

\[\eta_{ij}(t)=\frac{1}{d_{ij}} \tag{52}\]

where \(J\) represents the set of all neighboring nodes \(j\) of node \(i\); \(s\) represents the node that ant \(k\) is allowed to select next; \(\tau_{ij}\) represents the pheromone concentration of each pathway; \(\eta_{ij}\) is the heuristic function, as shown in Equation (52); \(Q_{ij}\) is as shown in Equation (48), \(\alpha\) is the pheromone concentration factor, \(\beta\) is the heuristic function factor, \(\gamma\) is the Q value factor, and \(\delta\) is the waiting time factor. ### Robust Multiobjective Optimization

The traditional robust selection strategy selects only the solution set with the highest feasibility. However, due to the conflicting nature among multiple objectives, it is difficult for the other objectives of the most feasible solution to be close to the optimum. For practical problems, appropriate delays are acceptable. So this paper proposes a robust selection method based on uniformly distributed weights in the solution space combined with Monte Carlo simulation. Since the solutions in the neighborhood centered at a certain point in the solution space have some degree of similarity, this robust optimization method splits the solution space by uniform weights, then converts the feasibility of solutions into Monte Carlo feasibility times. Finally, the optimal set of solutions in the solution space corresponding to each weight is found separately. This strategy obtains solutions that take into account robustness while guaranteeing diversity in multiple objectives. Through this strategy, the more feasible solutions can be retained, and at the same time, based on the uniformly distributed weights, the solutions with lower feasibility but better other objectives are also retained, so as to provide the decision maker with more decision-making options in the face of different practical needs. First, the solution space is partitioned by uniformly distributed weights, as shown in the Figure 8, with different colors representing different weights. Each weight represents a section of the solution space. Then, each solution is associated with the weight which is closest to it.