# Visual Tracking Using Wang-Landau Reinforcement Sampler
## 2 Related Work
### General Visual Tracking Methods  
Because of the representation power of deep neural networks [17; 18; 19], recent visual trackers have extracted useful features and considerably increased their accuracy [20; 21; 22; 23] referred to as deep learning-based visual tracking methods. [PERSON] et al. [23] presented the first visual tracker that adopted deep features, in which a stacked denoising autoencoder was used to extract generic features, and a classification layer was utilized to determine whether a current image patch is the foreground. [PERSON] et al. [21] considered visual tracking problems as binary classification problems and divided visual tracking videos into multiple domains to extract multidomain features. [PERSON] et al. [20] improved visual tracking accuracy by training deep neural networks using object recognition datasets and extracting hierarchical features. [PERSON] et al. [24] extracted deep features using the VGG-m network [25] and combined variational autoencoders with the particle Markov chain Monte Carlo method for multiple variable inferences.  
Siamese network-based visual trackers [26; 27; 28] transformed visual tracking problems into matching problems, in which exemplar patches were matched to search window patches through a cross-correlation operation. For this purpose, two deep neural networks were designed withsimilar architectures that share parameters. Because matching is typically faster than classification, Siamese network-based visual trackers have demonstrated superior speed. [PERSON] et al. [26] implemented a Siamese network using only convolutional layers for visual tracking. [PERSON] et al. [28] proposed a Siamese network-based visual tracker that can run at a real-time speed of 100 FPS. [PERSON] et al. [29] introduced a novel approach for visual tracking, in which no model updating was required. They argue that visual tracking accuracy is derived from a powerful matching function, which can be learned using a sufficiently large amount of training data. However, Siamese network-based visual trackers easily miss the targets if there are severe occlusions and background clutter. They lack both an explicit process to obtain feedback from the environment to recover erroneous trajectories and an exploration mechanism to sufficiently search the state space.  
To handle long-term video sequences, DASiam [30] considered larger search areas than conventional methods, if target objects have high confidence values. GlobalTrack [31] and SiamRPN [32] have explicit redetection processes, in which whole regions are searched to recover missed target trajectories. However, GlobalTrack and SiamRPN require high computational costs because these methods employ full-search strategies. In contrast, our proposed tracker presents an efficient exploration technique based on Wang-Landau sampling. Thus, our tracker is significantly faster than the aforementioned methods and runs in realtime.  
In contrast to the aforementioned methods, we incorporated reinforcement learning into visual tracking problems formulated as action-decision frameworks, in which the proposed method can recover erroneous trajectories using feedback from the environment and sufficiently explore the state space to capture abrupt target motions.