Paper:
Hierarchical Attentive Recurrent Tracking
Class-agnostic object tracking is particularly difficult in cluttered environments as target specific discriminative models cannot be learned a priori. Inspired by how the human visual cortex employs spatial attention and separate "where" and "what" processing pathways to actively suppress irrelevant visual features, this work develops a hierarchical attentive recurrent model for single object tracking in videos. The first layer of attention discards the majority of background by selecting a region containing the object of interest, while the subsequent layers tune in on visual features particular to the tracked object. This framework is fully differentiable and can be trained in a purely data driven fashion by gradient methods. To improve training convergence, we augment the loss function with terms for auxiliary tasks relevant for tracking. Evaluation of the proposed model is performed on two datasets: pedestrian tracking on the KTH activity recognition dataset and the more difficult KITTI object tracking dataset.

Review:
Summary: The authors present a framework for class-agnostic object tracking. In each time step, the presented model extracts a glimpse from the current frame using the DRAW-style attention mechanism also employed by Kahoue et al. Low level features are then extracted from this glimpse and fed into a two-stream architecture, which is inspired by the two-stream hypothesis. The dorsal stream constructs convolutional filters, which are used to extract a spatial Bernoulli distribution, each element expressing the probability of the object occupying the corresponding locations in the feature map. This location map is then combined with the appearance-based feature map computed by the ventral stream, removing the distracting features, which are not considered to be part of the object.
An LSTM is used for tracking the state of the object, with an MLP transforming the output of the LSTM into parameters for both attention mechanisms and an offset to the bounding box parameters in the previous time step.
The results on KTH and the more challenging KITTI dataset are significantly closer to the state of the art than previous attention based approaches to tracking.
Qualitative Assessment:
The paper reads well and is easy to follow. The description of the model is detailed enough for replication and experiments are performed providing evidence for some of the premises stated in the description of the model.
The results are impressive compared to previous work on attention based tracking. Since, you used a standard tracking benchmark, I think more performance numbers from the tracking community could have been included to show how close the presented method is to the state of the art. (example: Re3: Real-Time Recurrent Regression Networks for Object Tracking
D Gordon, A Farhadi, D Fox, arXiv preprint arXiv:1705.06368)