# End-to-End Deep Reinforcement Learning for Image-Based UAV Autonomous Control
## 4 Proposed Method
### Algorithm Implementation
The final height is below 0.18 m and the final lateral error is less than 0.5 m. The curves of the UAV states, including its linear velocities with reference to the world frame (\(v_{x},v_{y},v_{z}\)) and body rates (\(p,q,r\)) varying with position are shown in Figure 15. The velocities along the \(x\)-axis and the \(y\)-axis first increase and then decrease. Both the pitch rate and the yaw rate vibrate a little. The control commands of the six actuators varying with position are also shown in Figure 16. The sign of the left and the right aileron are opposite. In addition, the rudder provides a small negative deflection that steers the UAV towards the center of the runway to eliminate the initial lateral offset. #### 5.3.3 Case 3

For the third case, the initial position of the UAV is (\(-\)4.5, 28, 12) m. The initial deflection angle of the left and right ailerons, the elevator, and the flaps is \(-\)0.35 rad. In addition, that of the rudder is 0 rad. The initial velocity along the \(y\)-axis is \(-\)10 m/s. There is no disturbance outside. The border of the \(x\)-axis is [\(-\)10, 50] m and for the \(y\)-axis is [\(-\)30, 30] m. Screenshots taken during the landing and the corresponding processed images are shown in Figure 9:0

The landing trajectories are shown in Figure 9:1 They indicate that the UAV makes a turn under this condition and that the distance between the UAV and the centerline of the runway shrinks during its landing. The final height is less than 0.17 m and the lateral error is less than 0.3 m. The curves of UAV states including its global linear velocities (\(v_{x},v_{y},v_{z}\)) and body rates (\(p,q,r\)) varying with position are shown in Figure 9:2 The velocity of the \(x\)-axis first increases and then decreases. The velocity of the \(y\)-axis continuously decreases and the yaw rate increases during the adjustment of the heading direction of the UAV. The control commands of the six actuators varying with position are also shown in Figure 9:3 The rudder has a large negative deflection angle to generate the yaw moment. ### Discussion

The simulation results under different cases confirm that the proposed end-to-end deep reinforcement learning-based method is able to generate appropriate actuator control commands directly on the basis of the input image in order to perform landing in a fixed-wing UAV. After training, the UAV was able to handle the control pattern of landing from different initial points with a final height error within 0.2 m and a lateral error within 0.5 m. Case 1 was the basic situation, with zero lateral offset and zero heading angle offset. Case 2 proved that the UAV was able to learn to eliminate the initial lateral offset. In Case 3, although the UAV started with a large heading angle offset of nearly 90\({}^{\circ}\) and a relatively large distance away from the centerline of the runway of nearly 30 m, it was still able to gradually adjust its direction and land successfully. It is worth noting that the quantitative results reflect the general variation tendency of the UAV states during the landing process except for the last phases. The drastic fluctuation of the UAV states when the craft is close to the ground mainly results from the simulated aircraft model. The lack of landing

Figure 9:4 The control commands for the actuators under case 2.

gear means that part of the body always lands first, leading to irregular motion close to the ground. Additionally, the constraints to the final attitudes of the UAV maybe not strict enough. Figure 9:5 Cont. Figure 9:6 The UAV states under case 3. (**a**) global linear velocities. (**b**) body rates. Figure 9:7 The landing process under case 3 (**a**–**e**). (**a**) \(t=0.0\) s; (**b**) \(t=0.24\) s; (**c**) \(t=2.03\) s; (**d**) \(t=2.22\) s; (**e**) \(t=3.13\) s.

## 6 Conclusions

An end-to-end autonomous control method based on DRL is proposed in this paper and validated on the basis of several simulation experiments under the example case of fixed-wing UAV landing.