## 1 Introduction
### Related Works  
Some researchers have already applied RL algorithms to control aerial robots. However, the bulk of studies focused on directly using RL algorithms to control aerial robots. As an example of these efforts, in a research study published in [38], a deep reinforcement learning algorithm was employed to control a fixed-wing aerial robot. The nonlinearities of the dynamics model, as well as the coupling effect between lateral and longitudinal control, were taken into account in the mentioned research study. The Proximal Policy Optimization (PPO) is the RL algorithm utilized in the study. Similarly, another research paper [39] employed PPO to regulate the attitude of a quadcopter. A research study presented in [40] employed a control system to achieve accurate autonomous driving of an aerial vehicle while landing on a platform. In the latter research paper, the Deep Q-learning Network (DQN) is the cornerstone of the controlling approach to mapping the images of poor quality to control states. Although the DQN method appears to be a promising approach for tackling vision-based control problems, it has significant drawbacks that limit its usage in more advanced vehicle control tasks that rely on image processing. As another example of this series of research efforts, a research study published in [41], leveraged an RL-based approach to control an aerial robot with the objective of capturing photographs of a person's front view, particularly his face. RL-based controllers also have found their ways into the world of morphing aerial robots. For instance, a work presented in [42] used a combination of the PPO algorithm with a PID controller to control a morphing aerial robot.  
In contrast to the majority of the mentioned works that used RL algorithms to directly control aerial robots, some researchers opted not to use RL for that purpose. Instead, some attempts have been made to use the RL-based algorithm as a foundation for active tuning and state estimation mechanisms for other classical controllers. The drawback of a direct RL-based control algorithm is its slow response, compared with that of conventional PID controllers that provide very fast controlling responses [43]. In addition, direct deep reinforcement learning-based controlling approaches does not provide any analytical guarantees for the stable response and robustness in the control process, as unexplainable neural networks underpin its structure. However, RL has the potential to be used along with traditional control algorithms to provide adaptive and robust controllers.  
As an example of the latter approach, the research study reported in [44] employed a fault-tolerant RL-based adaptive controller that combined an RL-based adaptive algorithm (in the study PPO) with a PID controller and an Unscented Kalman Filter (UKF) to develop a fault-tolerant RL-based adaptive controller. The proposed controlling strategy has employed a hybrid of parameter estimation and a deep reinforcement learning method. When the value of the parameters associated with faults affected the controller performance, the algorithm updated the PID controller. Although the findings of the study revealed a satisfactory control response for altitude control, the attitude control response did not provide clear superiority over earlier conventional controllers. Another research study [45] developed an adaptive neuro-fuzzy PID controller for nonlinear systems based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) method. The observation of the environment is integrated with information from a multiple-input single-output (MISO) fuzzy inference system (FIS) and has a specifically defined fuzzy PID controller functioning as the actor in the TD3 method, which provides automated tuning of fuzzy PID controller gains.