## 2 Smoothing paths for 3D planning.
### Layer IV: Intelligent Decision-Making System for Autonomous UAVs Navigation
The use of this method in both trajectory planning and autonomous navigation of vehicles such as UAVs is increasingly widespread mainly due to the possibility of generating a model that allows, to a UAV, a specific objective by navigating unsupervised through unknown environments, of which no prior information is available and, where through information collected from the environment and the state of the aircraft, a success ful decision-making can be made. In this way, thanks to the knowledge acquired by the UAV through DRL methods such as DQN, complex tasks can be undertaken in dynamic environments such as forest and urban fires. This complex learning is made possible by being able to introduce to ANNs a set of high-dimensional observations that allow the agent to learn that policy that chooses, always, that action that maximizes the return value, that is, the set of rewards when performing a sequence of actions on the environment. Therefore, the main idea of the development proposed in this chapter focuses on being able to include, within the autonomous navigation architecture, a new layer that gives systems a sufficiently advanced AI so that each UAV chooses a sequence of actions that allows them to navigate from one path point to another avoid colliding with objects present in the environment through autonomous decision-making. The section shows the developments performed in this area and the results achieved, comparing them based on the training parameters used, the set of states, the set of observations and the environment used. For both algorithm training and results validation, a software architecture based on TensorFlow [41] and Keras [42] has been used integrating ROS and Gazebo, in such a way as to enable training on a simulated environment that facilitates, in the future, the move from implementation and validation to real aerial platforms. To establish a DRL model as versatile and complete as possible that allows a UAV to navigate, from one path point to another through an unknown environment, different trainings have been carried out in which aspects such as the structure of the ANN, the space of actions or training and learning parameters have been modified. What has remained constant throughout development are the system inputs, or what in DRL is known as observation space and which, in this case, is linked to the information coming from the UAV and, which allows to know the state of the UAV within the environment. The number of observations is 7 and corresponds to the \(X\), \(Y\), \(Z\) position of the UAV, its orientation at \(pitch\), \(roll\) and \(yaw\) angles, and finally the _distance_ value to obstacles obtained by using a sonar sensor. This set of parameters constitutes the entry to the ANN and allows to know the status of the UAV at each step, being able to know if it is approaching or moving away from the objective, whether the stability of the system is correct, whether the action taken has caused the UAV to acquire a compromised physical state or, finally, to know whether it is approaching or moving away from an obstacle present in the environment. As indicated, the observation space constitutes the input to the neural network model used during training and that, with changes in the number of layers and dimensions of the output space of each of them, presents a similar general structure in all cases, as shown in Figure 14, in the sense that the entry into the network will always be the observation space and the output of it will be characterized by the action space used at that time. The following discusses those cases in which a model capable of navigating the environment has been obtained, without supervision, from one given location to another, reaching the previously set goal.