# End-to-End Deep Reinforcement Learning for Image-Based UAV Autonomous Control
## 4 Proposed Method
### Algorithm Implementation
### Simulation Results

As shown in Table 1, the simulation experiments under three typical cases varying with respect to the initial pose of the UAV are conducted:

1. UAV starts landing from a point exactly aligned with the center of the runway;
2. UAV starts landing from a point parallel to the center of the runway with a small lateral offset;
3. UAV starts landing from a point vertical to the runway with a large lateral offset. The detailed description and analysis of the testing results under each condition are given in the following. #### 5.3.1 Case 1

For the first case, the initial position of the UAV is \((-25,0,12)\) m. The initial deflection angle of the left and right ailerons, the elevator, and the flaps of the UAV is \(-0.35\) rad. In addition, that of the rudder is 0 rad. The initial velocity along the \(x\)-axis is 10 m/s. There is no disturbance outside. The border of the \(x\)-axis is \([-30,50]\) m and for the \(y\)-axis it is \([-10,10]\) m. The screenshots during the landing and the corresponding processed images are shown in Figure 9.

The landing trajectories of the mass point of the UAV model are shown in Figure 10. The final height is less than 0.17 m and the lateral error is less than 0.04 m. The curves of UAV states, including its global linear velocities (\(v_{x},v_{y},v_{z}\)) and angular velocities with respect to the body frame \((p,q,r)\) varying with position, are shown in Figure 11. The \(y\)-axis velocity of the UAV is maintained at around 0 m/s, and the angular velocity of the \(x\)-axis vibrates at around \(-0.15\) rad/s before touching the ground. It is obvious that the states of UAV change dramatically during the last seconds. The main reason for this is the lack of landing gear, meaning that part of the body always lands first, leading to irregular motion close to the ground. And this phenomenon also occurs in other conditions. The control commands of the six actuators varying with position are also shown in Figure 12. It can be seen that the rudder exhibits little change from zero, which matches the fact that there is no need for the UAV to steer its yaw angle when it starts landing from the point that exactly aligned with the centerline of the runway. \begin{table}
\begin{tabular}{c c} \hline Case 1 & Exactly aligned with the centerline of the runway \\ \hline Case 2 & Parallel to the centerline of the runway with a small lateral offset \\ \hline Case 3 & Vertical to the runway with a large lateral offset \\ \hline \end{tabular}
\end{table} 
Table 1: The initial conditions of the three different cases. Figure 9: The landing process under case 1 (**a**–**e**). (**a**) \(t=0.0\) s; (**b**) \(t=1.08\) s; (**c**) \(t=2.10\) s; (**d**) \(t=3.07\) s; (**e**) \(t=4.12\) s.

Figure 11: The UAV states under case \(1\). (**a**) global linear velocities. (**b**) body rates. Figure 12: The control commands for the actuators under case \(1\). Figure 10: The landing trajectories under case \(1\). (**a**) 3D trajectory. (**b**) 2D trajectory in the \(x\)-\(y\) plane. #### 5.3.2 Case 2

For the second case, the initial position of the UAV is \((-25,\,-2,\,12)\) m. The initial deflection angle of the left and right ailerons, the elevator, and the flaps of the UAV is \(-0.35\) rad. In addition, that of the rudder is 0 rad. The initial velocity along the \(x\)-axis is about 10 m/s. There is no disturbance outside. As in case 1, the border of the \(x\)-axis is \([-30,50]\) m and that for the \(y\)-axis is \([-10,\,10]\) m. Screenshots taken during the landing and the corresponding processed images are shown in Figure 13.

Figure 13: Cont. The landing trajectories are shown in Figure 14. It is obvious that the UAV is not aligned with the center line of the runway at the beginning. However, it can adjust itself to land on the centerline of the runway.