# End-to-End Deep Reinforcement Learning for Image-Based UAV Autonomous Control
## 2 Related Work  
Traditional autonomous UAV control schemes are mostly based on the perception-planning-control framework. Ref. [20] focused on vision-based human following, wherein the researchers used a Haar cascade classifier to first detect the human face and track it with a Kalman filter. Then, the coordinate transformation was calculated to attain a pose estimation. In addition, an appropriate distance was maintained between them by using a linear PD controller. Ref. [21] proposed a system for the landing of a fixed-wing UAV comprising a vision module, a range finder (Lidar), and a high-precision aircraft controller. In addition, data from different sources are fused according to various phases. For quadrotor landing on a moving platform, [22] proposed a full-stack solution based on state-of-the-art computer vision algorithms, multi-sensor fusion for localization of the robot, detection and motion estimation of the moving platform, and path planning for fully autonomous navigation. An image-based visual servo controller was proposed in [23] for quadrotor vertical takeoff and landing of an unmanned aerial vehicle (UAV). The controller utilized an estimate of the flow of image features as the linear velocity cue and assumed angular velocity and attitude information available for feedback. In [24], binocular stereo vision was applied to perceive obstacles and the power lines to be inspected. In addition, the real-time local optimal path was then planned by combining the interfered fluid dynamical system (IFDS) method with the rolling optimization strategy. Ref. [25] formulated the position-based visual servo problem for a quadrotor equipped with a monocular camera and an IMU relying only on features on planes and lines in order to fly above and perch on arbitrarily oriented lines. Ref. [26] proposed a trajectory generation method for crossing narrow gaps in consideration of geometric, dynamic, and perception constraints. In addition, it also estimated the full state by fusing gap detection from a single onboard camera with an IMU. Similar methods were also discussed in [27; 28; 29] with respect to quadrotors equipped with VINS (visual-inertia navigation system), achieving safe flight through cluttered environments on the basis of the mapping module, the online trajectory planner, and the cascade controller. Ref. [30] provided a scheme for autonomous UAV guidance in a precision agriculture scenario based on image processing techniques. Crop row detection was performed on the basis of the Hough transformation to identify the line representing the crop row. Then, a line follower algorithm was designed to generate the trajectory for tracking the UAV. With the development and application of machine learning and neural networks, direct mapping between perception and control can be established, making the sensorimotor control system more intelligent, including in the context of autonomous control of UAVs. Ref. [31] used only a single camera to perceive the environment and train a reactive controller on the basis of imitation learning in accordance with demonstrations by a human pilot in order to avoid obstacles in a cluttered environment. By virtue of the DAgger technique, the reactive control policy mapped the visual features directly to the left or right velocity control input. In addition, the experimental results showed that under the trained policy, the UAV was able to fly through dense forest at a velocity of up to 1.5 m/s. Ref. [2] focused on the problem of learning UAV autonomous control for the purpose of tracking a moving target. A neural network consisting of convolutional layers and fully connected layers was used to generate the control command on the basis of the image input. In addition, a hierarchical approach combining a model-free policy gradient method with a conventional feedback controller was proposed. In [32], a fast eight-layer residual network named DroNet was proposed to safely drive the UAV through the streets of a city.