## 3 Methodology
### Genetic Algorithm
This poses a new and exciting question:

\"Can an optimal controller be replaced by a more advanced machine model such as Reinforcement Learning to learn a specific control law completely from scratch?\"

### Reinforcement Learning

Reinforcement Learning (RL) is a fast-growing area of machine learning that enables artificially intelligent agents to react to changes in an environment with the motivation of accumulating reward. Figure 16 presents the block diagram for the Reinforcement Learning process. The Agent makes decisions based on the policy, this policy is updated by a reinforcement learning algorithm based on rewards obtained for the previous action. Over time, the agent will learn to act in a way that maximizes its rewards from the environment. By introducing an agent to control the RUAV the controller is no longer required to define the conventional control laws as the agent learns them itself. However, by using a controller such as Linear Quadratic Gaussian over Reinforcement Learning, the implemented controller has no complexity and, therefore, no large computational overheads. In order to make reinforcement learning a viable option for machine learning control, the complexity of such models must be reduced to enable lightweight controllers to be created for real applications. Figure 15: Percentage Overshoot and settling time for best controllers. ## 7 Conclusions

This work has demonstrated that the transient response of optimal controllers can be improved by tuning using machine learning metaheuristic algorithms. Firstly, the conventional method of manual tuning was tested on well-known state-of-the-art methods such as Full State Feedback, Full State Compensator and Linear Quadratic Gaussian controllers. Secondly, metaheuristic algorithms (Particle Swam Optimization and Genetic Algorithm) were evaluated for their tuning capabilities for the same controller architectures. This demonstrated that improvements in performance are possible by choosing an algorithmic approach to tuning controller gains. The best controller design tested was the Linear Quadratic Gaussian tuned by Genetic Algorithm, this controller outperformed the same controller tuned by manual tuning, Particle Swarm Optimization, and a Genetic Algorithm Hybrid approach. The best controller (LQG-GA) had the fastest settling time outperforming manual tuning, Particle Swarm Optimization and GA-Hybrid by 20%, 17% and 21%, respectively, while attenuating measurement and process noise present in the system. There is a high computational cost associated with running optimization algorithms for tuning controller gains but as this is completed offline, the computational expense to the RUAV in flight remains the same. The key to successful implementation of PSO or GA is in the selection of an effective Fitness Function. Currently, there are still shortcomings associated with zero convergence criteria but this improvement in transient response induced by machine learning is a key step towards implementing safe, year-round flight of RUAVs in Northern Ireland. Conceptualization, [PERSON]; methodology, [PERSON]; software, [PERSON]; validation, [PERSON]; formal analysis, [PERSON]; investigation, [PERSON]; resources, [PERSON]; data curation, [PERSON]; writing--original draft preparation, [PERSON]; writing--review and editing, [PERSON] and M.U.H. ; visualization, [PERSON]; supervision, [PERSON]; project administration, [PERSON] and M.U.H. ; funding acquisition, M.U.H. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. The data presented in this study are available on request from the corresponding author. The authors declare no conflict of interest. Figure 16: Reinforcement Learning Block Diagram [26]. ## Appendix C Simulation Parameters for Full State Compensator