# Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
## 4 Experiments
### Training
#### 4.1.2 Implementation Details
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layers** & **Sub-Layers** & **Specification** & **Input Layer** \\ \hline \multirow{8}{*}{Spatial Features Encoder} & & Conv1a + ReLU + BN & k = 3, s = 1, p = 1 & Input image (64 \(\ast\) 64 \(\ast\) 3) \\  & & Avg-Pool & / & Conv1a + ReLU + BN \\  & Channel & Max-Pool & / & Conv1a + ReLU + BN \\  & Attention & Shared Conv2a + ReLU & k = 1, s = 1, p = 1 & \{Avg-Pool, Max-Pool\} \\  & Module & Shared Conv2b & k = 1, s = 1, p = 0 & Conv2a + ReLU \\  & & Sum & / & Shared Conv2b \\  & & Sigmoid & / & Sum \\  & & Plus1 & / & \{Sum, Conv1a + ReLU + BN\} \\ \cline{2-4}  & & mean & / & Plus1 \\  & Spatial & Concat & / & mean, max \\  & Attention & Conv3a & k = 3, s = 1, p = 1 & Concat \\  & Module & Sigmoid & / & Conv3a \\  & & Plus2 & / & \{ Plus1, Sigmoid \} \\  & & Conv3b + Sigmoid & k = 3, s = 1, p = 1 & Plus2 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Network architecture of the RL controller. ## References

* (1)
* (2) [PERSON]; [PERSON]; [PERSON]; [PERSON] Vision based ground object tracking using AR.Drone quadrotor. In Proceedings of the 2013 International Conference on Control, Automation and Information Sciences (ICCAIS), Nha Trang City, Vietnam, 25-28 November 2013; pp. 146-151. [CrossRef]





* [PERSON] et al. (2019) [PERSON]; [PERSON]; [PERSON] [PERSON] Flight test of Quadcopter Guidance with Vision-Based Reinforcement Learning. In Proceedings of the AIAA Scitech 2019 Forum,San Diego, CA, USA, 7-11 January 2019. * [PERSON] et al. (2019) [PERSON]; [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] Autonomous Vehicular Landings on the Deck of an Unmanned Surface Vehicle using Deep Reinforcement Learning. _Robotica_**2019**, _37_, 1867-1882. [CrossRef]
* [PERSON] et al. (2019) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] End-to-End Active Object Tracking and Its Real-World Deployment via Reinforcement Learning. _IEEE Trans. Pattern Anal. Mach. Intell._**2019**, _42_, 1317-1332. [CrossRef] [PubMed]
* [PERSON] et al. (2018) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Learning Unmanned Aerial Vehicle Control for Autonomous Target Following. In Proceedings of the 27 th International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 13-19 July 2018; pp. 4936-4942. * [PERSON] et al. (2013) [PERSON]; [PERSON]; [PERSON] V-REP: A versatile and scalable robot simulation framework. In Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3-7 November 2013; pp. 1321-1326. [CrossRef]
* [PERSON] et al. (2019) [PERSON]; [PERSON]; [PERSON] PyRep: Bringing V-REP to Deep Robot Learning. _arXiv_**2019**, arXiv:1906.11176. * [PERSON] (1994) [PERSON] _Markov Decision Processes: Discrete Stochastic Dynamic Programming_, 1 st ed. ; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 1994. * [PERSON] et al. (2015) [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON] Trust Region Policy Optimization. In Proceedings of the 32 nd International Conference on Machine Learning, Lille, France, 7-9 July 2015; pp. 1889-1897. * [PERSON] et al. (2017) [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] Proximal Policy Optimization Algorithms. _arXiv_**2017**, arXiv:1707.06347. * [PERSON] et al. (2015) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] [PERSON] High-dimensional continuous control using generalized advantage estimation. _arXiv_**2015**, arXiv:1506.02438. * [PERSON] et al. (2018) [PERSON]; [PERSON]; [PERSON]; [PERSON] Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8-14 September 2018; pp. 3-19. * [PERSON] et al. (2018) [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] Motion perception in reinforcement learning with dynamic objects. In Proceedings of the Conference on Robot Learning, Zurich, Switzerland, 29-31 October 2018; pp. 156-168. * [PERSON] et al.