## 4 Simulation Results and Discussion
Finally, the P-N reward provides better performance than the PID even with OR. Table 3 shows the mean value of the power output obtained by these experiments. The best value is obtained by combining OL-LR and \(\Delta\)MEAN-RS. OL-LR is the best policy update followed by SAR. O-R again provides the worst results. In this case, the best reward strategies are \(\Delta\)MEAN-RS and \(\Delta\)MSE-RS. This may be because the mean value measurement is intrinsically considered in the reward calculation. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**Policy Update**} \\ \hline Reward & OR & SAR & MAR & OL-LR & LF-LR & QL \\ \hline PRS & 123 & 126 & **123** & 123 & 123 & 125 \\ VRS & 193 & 125 & 152 & 137 & 156 & 174 \\ MSE\_RS & 124 & 124 & **123** & 123 & 124 & **124** \\ MEAN\_RS & **122** & **123** & 124 & 124 & **122** & **124** \\ \(\Delta\)MSE-RS & 183 & 127 & 130 & 122 & 159 & 150 \\ \(\Delta\)MEAN-RS & 180 & 124 & 149 & **121** & 168 & 165 \\ \hline PID & \multicolumn{6}{c}{273} \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Variance [kW] for different policy update algorithms and reward strategies. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**Policy Update**} \\ \hline Reward & OR & SAR & MAR & OL-LR & LF-LR & QL \\ \hline PRS & 402.86 & 407.59 & 403.22 & 404.11 & 402.28 & 406.53 \\ VRS & **306.71** & **265.62** & **271.56** & **266.37** & **281.39** & 287.83 \\ MSE-RS & 405.24 & 407.81 & 403.03 & 403.22 & 405.88 & 405.43 \\ MEAN-RS & 401.07 & 402.86 & 404.10 & 405.73 & 401.63 & 405.32 \\ AMES-RS & 308.49 & 270.21 & 273.08 & 274.71 & 282.84 & **274.33** \\ AMEAN-RS & 307.50 & 272.93 & 274.47 & 277.17 & 287.18 & **284.73** \\ \hline PID & \multicolumn{6}{c}{394.09} \\ \hline \hline \end{tabular}
\end{table} 
Table 2: MSE [W] for different policy update algorithms and reward strategies. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**Policy Update**} \\ \hline Reward & OR & SAR & MAR & OL-LR & LF-LR & QL \\ \hline PRS & **6.80** & 6.79 & 6.80 & 6.80 & 6.79 \\ VRS & 7.22 & 7.06 & 7.13 & 7.09 & 7.17 & 7.18 \\ ME-RS & 6.79 & 6.79 & 6.80 & 6.80 & 6.79 & 6.79 \\ MEAN-RS & **6.80** & 6.80 & 6.80 & 6.79 & 6.80 & 6.79 \\ AMSE-RS & 7.21 & 7.06 & 7.07 & 7.04 & **7.16** & **7.14** \\ AMEAN-RS & 7.21 & **7.05** & 7.12 & **7.03** & 7.17 & 7.16 \\ \hline PID & \multicolumn{6}{c}{7.25} \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Output power mean [kW] for different policy update algorithms and reward strategies. Table 4 presents the variance of the power output in the previous experiments. Unlike Table 2, in general, N-P rewards produce worse results than O-P rewards. This is logical because N-P rewards produce more change in the selected actions and a more varying output, therefore more variation. However, it is notable that the combination of VRS and SAR provides a good balance between MSE, mean value, and variance. Figure 3 represents the evolution of the saturated error and its derivative, iteration by iteration. In this experiment, the combination of SAR and \(\Delta\)MSE-RS is used. In Figure 3 it is possible to observe an initial peak of \(-1000\) W in the error (horizontal axis). This error corresponds to an output power of 8 kW (the rate power is 7 kW). It has not been possible to avoid it at the initial stage with any of the tested control strategies, even forcing the pitch to feather. A remarkable result is that, in each iteration, the errors are merged and centered around a cluster. This explains how the mean value of the output power approaches the nominal power over time. Figure 3: Distribution of the error when \(\Delta\)Mean Squared Error Reward Strategy (\(\Delta\)MSE-RS) and summation of all previous rewards (SAR) are applied for the first 25 iterations. Figure 8: Evolution of the MSE (**left**) and error centroid radius (**right**) for different reward strategies and update policy Learning and forgetting with learning rate (LF-LR).