Defining hand-crafted feature representations needs expert knowledge, requires time-consuming manual adjustments, and besides, it is arguably one of the limiting factors of object tracking. In this paper, we propose a novel solution to automatically relearn the most useful feature representations during the tracking process in order to accurately adapt appearance changes, pose and scale variations while preventing from drift and tracking failures. We employ a candidate pool of multiple Convolutional Neural Networks (CNNs) as a data-driven model of different instances of the target object. Individually, each CNN maintains a specific set of kernels that favourably discriminate object patches from their surrounding background using all available low-level cues (Fig.  1 ). These kernels are updated in an online manner at each frame after being trained with just one instance at the initialization of the corresponding CNN. Cue-4 Figure 1: Overall architecture with (red box) and without (rest) the classspecific version. Instead of learning one complicated and powerful CNN model for all the appearance observations in the past, we chose a relatively small number of filters in the CNN within a framework equipped with a temporal adaptation mechanism (Fig. 2) . Given a frame, the most promising CNNs in the pool are selected to evaluate the hypothesises for the target object. The hypothesis with the highest score is assigned as the current detection window and the selected models are retrained using a warm-start backpropagation which optimizes a structural loss function.
To handle long training times and a large number of training samples, DeepTrack REF employed a single CNN for learning effective feature representations of the target object by a purely online manner.