## 5 Conclusions and Future Works  
In this work, a RL-inspired pitch control strategy of a wind turbine is presented. The controller is composed by a state estimator, a policy update algorithm, a reward strategy, and an actuator. The reward strategies are specifically designed to consider the energy deviation from the rated power aiming to improve the efficiency of the WT.  
The performance of the controller has been tested in simulation on a 7 kW wind turbine model with varying different configuration parameters, especially those related to rewards. The RL-inspired controller performance is compared to a tuned PID giving better results in terms of system response.  
The relationship of the rewards with the exploration-exploitation dilemma and the \(e\)-greedy methods is studied. On this basis, two novel categories of reward strategies are proposed, O-P (Only-Positive) and P-N (Positive-Negative) rewards. The performance of the controller has been analyzed for different reward strategies and different policy update algorithms. The individual behavior of these methods and their combination have also been studied. It has been shown that the P-N rewards improve the learning convergence and the performance of the controller.  
The influence of the control parameters and RL configuration on the turbine response has been throughout analyzed and different conclusions regarding learning speed and convergence have been drawn. It is worth noting the relationship between the size of the reward and the need for forced exploration for the convergence of learning.  
Some potential challenges may include to extend this proposal to design model-free general purpose tracking controllers. Another research line would be to incorporate risk detection in the P-N reward mechanisms to perform safe non-forced exploration for systems, which must fulfill safety requirements during the learning process.  
As other future works, it would be desirable to test the proposal in a real prototype of a wind turbine. Also, it would be interesting to apply this control strategy to a larger turbine, and see if this control action affects the stability of a floating offshore wind turbine.  
Conceptualization, J.E.S.-G. and M.S.; methodology, J.E.S.-G. and M.S.; software, J.E.S.-G.; radial analysis, J.E.S.-G.; investigation, J.E.S.-G. and M.S.; resources, J.E.S.-G. and M.S.; data curation, J.E.S.-G.; writing--original draft preparation, J.E.S.-G. and M.S.; writing--review and editing, J.E.S.-G. and M.S.; visualization, J.E.S.-G. and M.S.; supervision, J.E.S.-G. and M.S.; project administration, M.S.; funding acquisition, M.S. All authors have read and agreed to the published version of the manuscript. This work was partially supported by the Spanish Ministry of Science, Innovation and Universities under MCI/AEI/FEDER Project number RTI2018-094902-B-C21.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{3}{c}{**KPI**} \\
**Reward** & **MSE [W]** & **Mean [kW]** & **Var [kW]** \\ \hline PRS & 404.10 & 6.80 & 124 \\ MSE-RS & 403.03 & 6.80 & 123 \\ MEAN\_RS & 404.81 & 6.80 & 124 \\ \hline VRS & 272.90 & 7.08 & 115 \\ \(\Delta\)MSE-RS & 267.91 & 7.07 & 135 \\ \(\Delta\)MEAN-RS & 269.61 & 7.08 & 137 \\ \hline PRS-VRS & 262.20 & 7.07 & 131 \\ VRS-MSE-RS & 265.50 & 7.05 & 127 \\ VRS-MEAN-RS & 270.30 & 7.04 & 117 \\ \hline PRS-\(\Delta\)MSE-RS & 264.99 & 7.80 & 125 \\ MSE-RS-\(\Delta\)MSE-RS & 284.39 & 7.00 & 111 \\ MEAN-RS-\(\Delta\)MSE-RS & 274.10 & 7.03 & 118 \\ \hline PRS-\(\Delta\)MEAN-RS & 266.61 & 7.04 & 119 \\ MSE-RS-\(\Delta\)MEAN-RS & 280.54 & 7.01 & 115 \\ MEAN-RS-\(\Delta\)MEAN-RS & 285.60 & 7.00 & 116 \\ \hline \hline \end{tabular}
\end{table}
Table 5: MSE, mean value and variance for different combinations of reward strategies.  
**Acknowledgments:** An earlier version of this paper was presented at (21 st International Conference on Intelligent Data Engineering and Automated Learning--IDEAL 2020) [36].  
**Conflicts of Interest:** The authors declare no conflict of interest.