## 4 Proposed Algorithm
### Different Parameter Settings  
In the previous experiments, this paper did not make any serious attempt to find the optimal parameter settings for NSACOWDRL. For the DDQN, the type of neural network affects the performance of the algorithm. In this section, based on the proposed model, the neural networks of the DDQN are set as the back propagation (BP), radial basis function (RBF), the general regression network (GRNN), and product-based network (PNN), respectively, so as to test the effects brought by different networks.  
Table 5 shows the performance of the algorithm with different networks. Figure 16 shows the \(p\) value of the metrics. By comparison, it can be seen that PNN has the best convergence and diversity, PNN has the closest performance to GRNN, and RBF has the worst performance. At the same time, as the interference coefficient increases, the variability between different networks begins to decrease.  
For illustration, Figure 17 shows the set of Pareto solutions in the approximate PF obtained by each algorithm with different interference coefficients. The interference coefficient increases from left to right and then from top to bottom in the subplots of Figure 16.  
Figure 16: \(p\) value heatmap of networks.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**HV**} & \multirow{2}{*}{\
\begin{tabular}{c} - \\ \(\zeta\) \\ \end{tabular} } & **NSACOWDRL** & **NSACOWDRL** & **NSACOWDRL** & **NSACOWDRL** \\  & & **BP** & **-RBF** & **-PNN** & **-GRNN** \\ \hline \multirow{8}{*}{Middle} & \multirow{2}{*}{Small} & 6.215E\(-\)01 & 6.208E\(-\)01 & 6.367E\(-\)01 & 6.427E\(-\)01 \\  & & 8.768E\(-\)04 & 9.602E\(-\)04 & 1.183E\(-\)03 & 5.848E\(-\)04 \\  & & 6.037E\(-\)01 & 5.901E\(-\)01 & 6.068E\(-\)01 & 6.026E\(-\)01 \\  & & 2.763E\(-\)04 & 2.104E\(-\)04 & 1.639E\(-\)04 & 1.809E\(-\)04 \\  & & 4.874E\(-\)01 & 4.732E\(-\)01 & 4.763E\(-\)01 & 4.901E\(-\)01 \\  & & 7.729E\(-\)04 & 2.365E\(-\)04 & 5.044E\(-\)04 & 4.079E\(-\)04 \\ \hline IGD & \% & & & & \\ \hline \multirow{8}{*}{Middle} & \multirow{2}{*}{
\begin{tabular}{c} Large \\ \end{tabular} } & 6.881E\(-\)02 & 7.188E\(-\)02 & 6.537E\(-\)02 & 6.816E\(-\)02 \\  & & 3.990E\(-\)05 & 1.428E\(-\)05 & 2.453E\(-\)05 & 1.668E\(-\)05 \\  & & 6.906E\(-\)02 & 6.994E\(-\)02 & 6.620E\(-\)02 & 6.826E\(-\)02 \\  & & 2.326E\(-\)05 & 2.518E\(-\)05 & 8.431E\(-\)06 & 1.017E\(-\)05 \\  & & 7.867E\(-\)02 & 7.937E\(-\)02 & 7.560E\(-\)02 & 7.539E\(-\)02 \\  & & 1.390E\(-\)05 & 9.746E\(-\)06 & 1.416E\(-\)05 & 1.164E\(-\)05 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Mean and variance metric values of HV and IGD for different networks.  
As the interference coefficient increases, each algorithm has more Pareto solutions which are more widely distributed in the solution space, and the variation in the distribution of solutions between algorithms begins to decrease. The figure shows that the solutions of NSACOWDRL_PNN are distributed at all levels between (0, 1) on the \(1/alpha\) axis in all subplots, while the solutions of NSACOWDRL_RBF are concentrated in two blocks. In terms of solution space distribution, the solution distribution obtained by NSACOWDRL_PNN is comparable to NSACOWDRL_GRNN which is better than the other algorithms.  
From these results, it can be shown that when adjusting the parameters of the algorithm, choosing the appropriate networks can effectively improve the performance of the algorithm. The appropriate neural network is selected by the type of problem, the Convolutional Neural Network (CNN) can be used to deal with discrete action space problems, while GRNN can be used to deal with nonlinear problems, which requires different network architectures and tuning for different problems.