# Integration of the 3D Environment for UAV Onboard Visual Object Tracking
## 3 Single Visual Object Tracking Pipeline for UAV
### Visual Appearance Modeling of the Object  
In this work, we rely on two state-of-the-art visual trackers, ATOM [27] and DiMP [28]. The chosen trackers achieve top ranks against numerous participants on general visual tracking benchmarks [2; 4; 6; 7; 10; 12; 29]. Figure 5 shows the main components of a standard pipeline for single object tracking, leveraged by methods such as ATOM and DiMP.  
ATOM includes three main components: A (1) _Feature Extractor_, a pretrained neural network--e.g., ResNet18, ResNet50 [57]--extracting salient features. A (2) _Classification_ component, composed of a two-layer convolutional neural network, which is trained during the tracking process to learn an appearance model of the object. The _Classification_ component proposes an initial estimation of the location and size of the object in the image as a bounding box. Lastly, a (3) _Bounding Box_  
Figure 4: The architecture of the proposed single visual object tracking framework. The visual tracker outputs a similarity score map inferred in the search area. The similarity score map and depth map approximation corresponding to the image are processed by the 3D Context component, estimating a new position for the object in the image.  
_regressor_ component, based on the IoU-Net [58] (trained offline), which refines the initially proposed bounding box.  
DiMP is a successor of ATOM and builds on the same elements. The main difference lies in the extension of the _Classification_ component. A new strategy for the initialization of the appearance model is used, expressing the appearance model with better weights at the start. The online learning process for updating the appearance model is also refined for faster and more stable convergence.  
For each incoming frame, a search area is created to delimit the possible positions of the object in the frame. The positioning and size of the search area depend on the previous estimated position and size of the object. Based on the features extracted from the search area and the appearance model of the object, a similarity score map is inferred, reflecting the resemblance of the extracted features with the learned appearance model. The highest score in the similarity score map is designated as the position of the object due to the maximum-likelihood approach. The object estimation module--i.e., IoU-Net--is used to identify the best fitting bounding box and thus, to refine the estimated position. To cope with changes in the appearance of the object--e.g., illumination variations, in-plane rotation, motion blur, background clutter--the appearance model of the object has to be adapted during tracking. This adaptation is reached by updating the appearance model regularly in ATOM and DiMP. The updates occur every 10 valid frames for ATOM and every 15 valid frames for DiMP. A valid frame corresponding to a frame where the object has been identified correctly--with a high similarity score. The new appearance model is adapted by retraining the classification component with the search areas of those valid frames. The tracker deals with distractors by recognizing multiple peaks in the similarity score map and immediately updating the appearance model of the object with a high learning rate.