# Hierarchical Active Tracking Control for UAVs via Deep Reinforcement Learning
## 4 Experiments
### Training
#### 4.1.2 Implementation Details
The path of the dynamic target is fixed, and the distance between the start and the end positions is increased from 10 m to 25 m.

The framework of the baseline controllers is presented in Figure 7, which corresponds to a three-stage method. The baseline controller obtains the bounding box of the dynamic target, using the passive tracker, and calculates the desired speed commands to pull the bounding box to the center of the image with a high-level PID controller. Then, the other low-level PID controller is used to track the speed commands. We do not use DNN-based passive trackers, such as SiamRPN [15], for the perception at baseline, considering that they need additional training with manual ground-truth labeling and that their model parameters are much larger than ours (the parameters of the backbond network ResNet-50 that are usually used in DNN-based trackers are 25.5 M, while the total parameters of our end-to-end perception and high-level decision-making controller are only 0.4 M). Instead,

Figure 5: Comparison of the learning curves with different control frameworks and auxiliary losses. (**a**) Learning curve of the end-to-end active tracking controller with or without auxiliary losses. (**b**) Learning curve of the hierarchical active tracking controller with or without auxiliary losses. two state-of-the-art passive trackers KCF [12] and MIL [13] are used as the perception components of the baseline controllers. In the test scenario, we apply different randomness to the color and speed of the dynamic target. The results of the validation experiments are presented in Figure 8. The bars show the results averaged over 10 episodes in the unseen test scenario. The delimiters show the maximum and minimum performance. These results suggest that our method achieves significantly better performance than the compared baseline methods under different randomness conditions. The method that we propose is robust to the dynamic target's appearance and velocity. The failure of traditional passive trackers in a few steps is probably due to the resolution of observations and the dynamics of the quadrotor. The observation with dimension (64 \(\times\) 64) is large for the high-level RL controller, but it is still not sufficiently informative for the passive tracker to update the bounding box precisely. Moreover, the dynamics of the quadrotor and the observations are coupled. The movement of the quadrotor is realized by changing its altitude, and then the target's position in the observation also changes, which complicates stable active tracking; however, our method can deal with these problems. #### 4.2.2 Analysis of Simulation Results

The spatial and temporal feature encoders in the RL controller are designed to encode the dynamic target's spatial and temporal features, respectively. To evaluate these encoders, we visualize the output actions of the RL controller for a test episode in the unseen scenario, where the dynamic target changes its color and speed at every timestep. Figure 6: The unseen test scenario. Figure 7: The control framework of our baseline controllers. The mapping from the ground truth distance error \(\delta\rho=\rho-\rho_{d}\) to the output normalized linear velocity \(\bar{V}_{fb}\) is shown in Figure 9a. In general, the correlation between \(\rho\) and \(\bar{V}_{fb}\) is positive, according to the fitting curve. However, the correlation is slightly weak, due to the random velocities of the dynamic target. To maintain the dynamic target at a fixed distance \(\rho_{d}\), the quadrotor must change speed frequently. Figure 9b shows the mapping from the yaw angle error \(\delta\varphi=\varphi\) to the normalized yaw speed \(\bar{V}_{yaw}\). We can observe that when the dynamic target is to the right of the quadrotor (\(\delta\varphi<0\)), the RL controller tries to force the quadrotor to yaw right (\(\bar{V}_{yaw}<0\)). Similarly, the RL controller outputs the command to yaw left when the target is to the quadrotor's left.