## 1 Introduction
### Control Problem of Aerial Robots
The controller, however, does not provide a good response for attitude control when there is too much variation in the parameters of the dynamical system. The problem stems from the fact that the prediction model in the linear MPC uses a linearized model around its ideal working point. The linearized model, however, is not sufficient to be used as a prediction model for the attitude control of aerial robots that need to do challenging maneuvers. Although the LMPC is not an ideal controller for attitude control, it is sufficient for altitude control, because most aerial robots fly at low altitudes with limited variation in elevation; thus, the linearized model around that working altitude can cover the behavior of a flying robot [24, 25]. Although PID controllers cannot fully cover fluctuations in system dynamics, they provide fast control response, which is a crucial ability for an aerial robot that requires agile attitude control responses to avoid obstacles and perform demanding maneuvers (e.g., delivery aerial robots in urban areas) [26]. In previous research efforts, many researchers have explored a variety of techniques to mitigate the problem in conventional standard PIDs. Several studies combined online tuning approaches with PID control to lessen the impact of changes in vehicle dynamics and disturbances [27]. A number of researchers opted for training neural networks to actively update the PID gain values of aerial drone controllers [28, 29]. However, neural network training needs a large database of labeled data. Another widely used approach to actively tune the PID control is the fuzzy-logic-based auto-tuning algorithm [30, 31, 32]. The drawback of the fuzzy-logic-based tuning strategy is that the efficiency of algorithm is strongly reliant on the fuzzy rules and the inference system set by the designer. However, some fluctuations in the system dynamics may be unanticipated, resulting in the generation of inaccurate control gain coefficients by the fuzzy logic [33]. In addition to the aforementioned optimal control and active tuning approaches (for PIDs), Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) are other new approaches that have recently made their way into the field of aerial robots control. RL and DRL do not require an accurate model of the plant under control or human designed control rules, in contrast to fuzzy logic that depends on the expert's expertise. As a result, these machine learning approaches have attracted the attention of many academics working on the control of systems with uncertain dynamical models [34, 35, 36]. Unlike neural network-based control techniques, deep reinforcement learning does not require large labeled datasets. This is a major benefit, since labelling data for all critical scenarios become more and more expensive as the number of required data grows. Another important advantage of deep reinforcement learning for control is its capability to directly map image features to control states, thereby resolving the need for the implementation of complex state estimators and image processing algorithms in some special cases [37].