Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: 
= Summary
This paper proposes to learn a visual tracking network for an object detection loss as well as the ordinary tracking objective for enhancing the reliability of the tracking network. The main motivation is that, current state-of-the-art models based on the Siamese architecture often blindly predict the center of a search window as target location due to the bias in datasets and training strategies. This issue is alleviated in this paper by introducing an auxiliary task, target detection in the entire image space. The auxiliary task is conducted by another branch on top of the visual feature shared with the tracking branch. By learning to detect object in the entire image space, the shared feature extractor will be trained to capture discriminative and unique appearance features of target.


= Decision
Although the main motivation is convincing and the manuscript is well written, I would recommend to reject this submission mainly due to its limited contribution and weakness in experimental analysis. 

(1) In the experiments, the practical benefit of adding the auxiliary detection task is demonstrated, but the final scores of the proposed model are clearly below those of current state of the art in terms of both reliability and accuracy. Further, it is not explained why the proposed model is worse than the other models in performance and what can be claimed as an advantage of the proposed method even in this situation. Also, I do not understand why the proposed model is not based on the current state of the art like SiamRPN++ but is built upon a manually designed/low-performance model. 

(2) The experiments in Section 5 do not demonstrate the advantage of the proposed model at all. In Figure 6 and 7, the difference between the proposed model and its reduced version without the auxiliary task looks quite subtle, and it is hard to say which one is better than the others. In Figure 8, adding the auxiliary detection task results in even worse tracking performance. 

(3) More qualitatively and quantitatively analysis should be done on the other tracking benchmarks and be compared with other tracking models recently proposed too.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: The paper examines the performance of Siamese single-object trackers. The authors claim that state-of-the-art Siamese trackers mostly rely on saliency detection in the center of the search window while ignoring the target instance representation and propose an additional object detection branch during training to mitigate this effect.

Strengths:
+ Analysis how perturbation target and search image influence tracking performance
+ Demonstrates that including detection objective during training improves performance

Weaknesses:
- Makes claims about Siamese trackers in general but only experiments with authors’ architecture
- Analyses in Figs. 6–8 not very helpful/conclusive
- No test case where capturing appearance is important for tracking and authors’ approach helps
- No comparison of tracker trained without detection objective on real vs. random targets

The paper is well motivated and has a clear hypothesis. However, its execution unfortunately leaves a lot of room for improvement and getting it into acceptable shape would require a major revision. Some details on my main criticisms:

Claim is way too general. The authors state that Siamese trackers in general suffer from the center bias problem. However, the authors do not analyze any other trackers than their own one. I understand that training a state-of-the-art tracker with the additional object detection branch could possibly require resources beyond what’s available to the authors. However, it’s not clear to me why the authors do not use the pre-trained version of at least one or two existing trackers to demonstrate their shortcomings (like Fig. 4)

Analyses in Fig. 6–8 are not very helpful. The only real effect the detector shows in Figs. 6–8 is a tiny improvement in IoU in Fig. 8 – otherwise the analyses neither support the claim nor do they reveal why detection as an additional objective actually helps. To establish that there is a center bias and trackers ignore the appearance term, the following two analyses could be done:

- Identify a set of test cases where capturing appearance is important (e.g. temporary occlusion) and demonstrate that your approach improves performance on these cases.

- For the table in Fig. 5, also show how a conventional tracker (i.e without object detection branch) performs on random targets. If the hypothesis is correct that the object detection objective during training reduces center bias and increases reliance on appearance, then a conventional tracker should show a smaller reduction due to random targets than your improved tracker does. The same quantitative analysis could also be done for existing state-of-the-art trackers, as it does not require training. 
 
Minor Comments:
- Caption of Fig. 4: is instead of if?
- Figure 5
	- y-axis label: lower is better despite “robustness” suggesting the opposite
- Why do the numbers in the figure not match those in the table?
- What is the center bias baseline, i.e. what is the performance if center of search image is predicted without any network?

review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper investigates representations learned by Siamese trackers. The paper argues that existing trackers rely on saliency detection despite being designed to track by template matching in feature space. An auxiliary detection task is proposed to induce stronger target representations in order to improve tracking performance. Experiments are performed on VOT2018 tracking dataset.

The paper investigates an interesting and active research problem of stronger object representations for deep visual object tracking. However, the proposed solution of just integrating an additional detection task branch within the Siamese tracking architecture is naive. The main idea of integrating instance driven detection as an auxiliary task is borrowed from [1].  [1] also utilizes a Siamese architecture that is similar to the ones generally used in visual object tracking to localize particular instances of objects. Therefore, the novelty of the proposed tracking approach is limited.

Some recent works, such as [2, 3] have also investigated a similar problem of richer object representations for deep visual tracking. These approaches are desired to be discussed and empirically compared in order to fully validate the strength of the proposed approach. 

The paper shows some qualitative analysis. However, most of it is limited to just few frames of an image sequence. Tracking datasets, such as VOT and OTB, provide additional analysis tools (i.e., attribute analysis) to thoroughly evaluate visual trackers. Such analysis is missing in the paper. For instance, the main argument of this paper is that current approaches rely on center saliency and likely struggle in the presence of occlusion. How does the proposed approach fare, compared to SOTA, on the subset of VOT image sequences that are labeled with occlusion?

On page 3, it is stated that "Our model uses a lightweight backbone network (MobileNetV2), and is somewhat simpler than recent state-of-the-art models  .....................  Although the model doesn’t outperform state-of-the-art, it attains competitive performance." The reviewer does not fully agree with this statement. A comprehensive empirical evaluation is crucial to fully access the merits of the contributions. State-of-the-art visual object trackers [2, 3, 4] achieve competitive tracking performance while being computationally efficient and fast. Therefore, a proper state-of-the-art comparison is desired to compare the proposed tracker with SOTA methods. Further, currently experiments are only performed on the VOT2018 dataset. The reviewer recommends to perform additional experiments on other large-scale datasets, such as TrackingNet [5] and Lasot [6] and compare the performance with SOTA methods that are also investigating the problem of richer object representations for tracking. 

[1] Phil Ammirato, Cheng-Yang Fu, Mykhailo Shvets, Jana Kosecka, Alexander C. Berg: Target Driven Instance Detection. CoRR abs/1803.04610 (2018).
[2] Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, Michael Felsberg: ATOM: Accurate Tracking by Overlap Maximization. CVPR 2019.
[3] Goutam Bhat, Martin Danelljan, Luc Van Gool, Radu Timofte: Learning Discriminative Model Prediction for Tracking. CoRR abs/1904.07220 (2019).
[4] Bo Li, Wei Wu, Qiang Wang, Fangyi Zhang, Junliang Xing, Junjie Yan: SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks. CVPR 2019.
[5] Matthias Müller, Adel Bibi, Silvio Giancola, Salman Al-Subaihi, Bernard Ghanem: TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild. ECCV  2018.
[6] Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, Haibin Ling:
LaSOT: A High-Quality Benchmark for Large-Scale Single Object Tracking. CVPR 2019.


review_title: Official Blind Review #2

Meta-review: This paper proposes to learn a visual tracking network for an object detection loss as well as the ordinary tracking objective for enhancing the reliability of the tracking network.  The reviewers were unanimous in their opinion that the paper should not be accepted to ICLR in its current form.  A main concern is that the proposed method shows improvement over a relatively weak base system.  Although the author response proposed to include additional analysis, but the reviewers felt that without the additional analysis already included it was not possible to change the overall review score.
Decision: Reject