Globally Optimal Object Tracking with
Complementary Use ofSingle Shot Multibox Detector andFully Convolutional Network
Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, and Seiichi Uchida
Kyushu University, Fukuoka, Japan

Abstract. Object tracking is one of the most important but still difficult tasks in computer vision and pattern recognition. The main difficulties in the tracking task are appearance variation of target objects andocclusion. To deal with those difficulties, we propose a object tracking
method combining Single Shot Multibox Detector (SSD), Fully Convolutional Network (FCN) and Dynamic Programming (DP). SSD and FCN
provide a probability value of the target object which allows for appearance variation within each category. DP provides a globally optimal
tracking path even with severe occlusions. Through several experiments,
we confirmed that their combination realized a robust object tracking
method. Also, in contrast to traditional trackers, initial position and atemplate of the target do not need to be specified. We show that the
proposed method has a higher performance than the traditional trackers
in tracking various single objects through video frames.
Keywords: Object Tracking, Single Shot Multibox Detector, Fully Convolutional Network, Dynamic Programming

1

Introduction

Object tracking is defined as the problem of estimating the spatio-temporal trajectory of a target object in an image. Although it has been studied for many
applications such as bio-image analysis, scene surveillance, autonomous vehicle
control, etc, it is still a difficult problem. One difficulty comes from appearance
variation. For example, for a general person tracking problem, we need to deal
with various clothes, poses, and body shapes under various illumination condition. Traditional methods assume a predefined template of the target object
and update it accordingly to any changes in appearance [1, 2]. Another difficultyis occlusion. Traditional object tracking methods are often intolerant to severe
occlusion [3–5].
In this paper, we propose a object tracking method robust to both of appearance variation and occlusion by using a complementary combination of Single
Shot Multibox Detector (SSD) [6], Fully Convolutional Network (FCN) [7], andDynamic Programming (DP) [8–10]. SSD and FCN are employed for tackling

2

Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, Seiichi Uchidaappearance variation. They have been proposed recently for object detection andthey can provide a probability value of a target object for each category, such as
person, car, and motorbike, given each bounding box or pixel respectively. Since
SSD and FCN are types of CNNs, large amounts of training samples will makethem robust to a variety of appearances.
To deal with occlusion, we utilized DP for global optimization of a target
object’s trajectory. DP is one of the most fundamental optimization techniquesand has been used for obtaining a globally optimal tracking path. Since a slope
constraint of DP prohibits the tracked position from moving steeply over all
frames, it is possible to obtain a stable tracking path, regardless of occlusion.
It is very important to note the reason why we use the two CNN-based object
detectors, SSD and FCN, in a complementary manner, is because they provide
detection results in different ways. SSD provides an accurate detection result toa clear target object, however it is impossible to provide a detection result in an
unstable situation such as occlusion. In contrast to SSD, FCN provides a result,regardless of any situation. Namely, combination of SSD and DP is useful tostable situation to obtain accurate result and it of FCN and DP is utilized in
unstable situation to obtain any result.
It is also noteworthy that the proposed method requires neither the initial
position nor the template of the target object. Traditional trackers may be sensitive to the template of the target object and the initialization in which theinitial position of the target object is denoted on the first frame. However, theproposed method does not require either the template nor the initialization due
to the synergy combining SSD, FCN, and DP.
The contributions of this paper are as follows. First, we show proposed
method achieved the highest accuracy compared to the traditional trackers introduced in the Visual Tracker Benchmark [11]. Second, we confirm that the complementary use of the two CNN-based object detectors, SSD and FCN, are usefulfor tracking. Third, we confirm that the proposed method tackles appearance
variation and occlusion through several experiments even without initialization,
templates, and modifying parameters.
The remaining of this paper is organized as follows. In Section 2, we introduce related traditional tracking research. Section 3 elaborates on SSD, FCN,
and DP and details the proposed method. In Section 4, we confirm that the
proposed method is a robust tracker through several experiments and analyze
the experimental results. Finally, Section 5 draws the conclusion.

2

Related Work

Object tracking is one of the important techniques in computer vision and has
been actively studied for decades. Most object tracking algorithms are divided
into two categories: generative and discriminative methods. Generative methods describe appearance of a target object using a generative model and search
for the target object region that fits the model best. A number of generativemodel based algorithms have been proposed such as sparse representation [12,Globally Optimal Object Tracking with SSD and FCN

3Table 1. Comparison between the proposed method and FCNT [29].
Tracker Target Template
Initialization Path optimization
Offline/Online
Proposed General Unnecessary Unnecessary Globally Optimal by DP Offline
FCNT
Specific Necessary Necessary
GreedyOnline13], density estimation [14, 15], and incremental subspace learning [16]. On contrary, discriminative methods build a model to distinguish a target object fromthe background. These tracking methods include P-N learning [17] and online
boosting [18–20]. Even though these approaches are satisfactory in restricted situations, they have inherent limitations which include occlusion and appearance
variation such as illumination changes, deformation etc.
To deal with limitations which traditional trackers can not tackle, recent
trackers employ Convolutional Neural Networks (CNN) [21, 22] and Deep Convolutional Neural Networks (DCNN) [23, 24] by focusing on their powerful performance. A number of trackers using neural networks have been proposed suchas human tracking, hand tracking, etc. [25–28]. Representative tracker using
a neural network is a Fully Convolutional Network based Tracker (FCNT) [29]which also utilizes FCN. This method utilizes multi-level feature maps of a VGG
network [30] to complement drastic appearance variation and distinguish a target object from its similar distracters. It selects discriminative feature maps anddiscards noisy ones, because the CNN features pretrained on ImageNet [23] arefor distinguishing generic objects. Even though FCNT achieved a high accuracy
compared to conventional trackers, initialization and templates are necessary to
track a target object.
Table 1 shows the comparison of characteristics between the proposed method
and FCNT. The main difference between the proposed method and FCNT is
whether initialization and templates of a target object are necessary or not.
Namely, FCNT can track only a specific target object with defined initial position and template. In contrast, it is possible to use the proposed method without them.. The other difference is that FCNT uses a greedy tracking algorithm
whereas the proposed method utilizes DP for globally optimal tracking. In Section 4.3, we will prove experimentally that the proposed method has superiority
over FCNT.

3

The Proposed Method

3.1

Likelihood Maps by Single Shot Multibox Detector andFully Convolutional Network

Fig. 1 shows the pipeline of how to obtain a likelihood map from input image.
In the proposed method, SSD and FCN are used for obtaining likelihood maps,
each of which shows a two-dimensional probability distribution of a target object

4

Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, Seiichi Uchida

Fig. 1. Pipeline to generate a likelihood map from an input image: (a) shows thepipeline to generate a likelihood map by SSD. The likelihood map by SSD provides
accurate probability values and positions of target objects when the targets are rather
easy for detection. When SSD fails to detect the target, we switch to FCN and employ
the likelihood map by FCN, as shown in (b). The likelihood map by FCN might include
noisy probability values, compared to it by SSD.

position at a certain frame. A peak in a likelihood map at frame t suggests a
candidate position of the target object at t. We will switch two likelihood maps
according to the situation as shown in Fig.1. This is because SSD and FCN
shows different behaviors especially when object candidate detection is difficult,as follows.
SSD is based on VGG-16 network which includes 13 convolution layers and3 pooling layers. It possesses supplementary two characteristics: convolutional
predictors and multi-scale feature maps. The convolutional predictors generatea probability value for the presence of each object category in each default box
and produce adjustments to the box to match the object shape. Additionally,
the network combines predictions from multi-scale feature maps with different
resolutions to handle objects of various sizes. We generate a likelihood map
by setting a probability value, on the center position of resulting bounding box.
Thus, likelihood maps obtained by SSD contain a very accurate probability value.
However, when SSD fails to detect a target object, likelihood maps can not be
obtained.
FCN is composed entirely of convolutional layers based on VGG-16 andtrained end-to-end, pixels-to-pixels, for classification and segmentation. It takes
input of arbitrary size and produces a correspondingly-sized likelihood map by
up and down sampled pooling layers. The likelihood map by FCN might include
noisy probability values by up and down sampled pooling layers. To obtain accurate positive response of a target object, links between the low-level fine layers
and the high-level coarse layers are constructed. These are so called skip con-

Globally Optimal Object Tracking with SSD and FCN

5nections which combines information from fine layers and course layers. Even if
we can obtain a more accurate likelihood map by skip connections, a likelihood
map obtained by FCN still includes noisy probability values, compared to SSD.
Using both SSD and FCN to obtain a likelihood map increases the tracking
accuracy. Although SSD is a detection method with high accuracy, it might notdetect the target object in unstable situations such as occlusion, blurriness, anddeformation, as shown in (b) of Fig. 1. If SSD fails to detect, we switch to FCN
and obtain likelihood maps by FCN. The success and failure criteria of detection
by SSD is whether the detected position exists within N pixels from the highest
value position of the previous frame or not. FCN provides likelihood maps for all
input images, regardless of unstable situation, even if they might include noisy
probability values. Note that, as discussed later, even when both SSD and FCN
cannot obtain likelihood values (e.g. when an object that has been tracked leaves
the scene), DP complements the tracking path.
The other merit of using SSD and FCN is their computational efficiency. A
naive method to obtain likelihood maps is to apply a CNN to a sliding window
region of an input image. Using this method requires many forward calculations
and can not deal with a target object of various sizes, because it accepts a fixedregion size as input. However, both SSD and FCN accept the entire image andonly require a single forward calculation with handling various sizes.
3.2

Global Path Optimization by Dynamic Programming

To apply DP to our method, we start by creating likelihood maps of each of theframes using SSD or FCN. Fig. 2 shows the procedure to obtain the most optimal
tracking path by DP. For each pixel on a likelihood map, we find the highest
value within a given slope constraint of the previous frame which prohibits frommoving steeply and create cumulative DP maps. This process is continued by
iterating over all of the frames. Cumulative DP map D(f ) is defined as:D(f ) (x, y) =max

x−ws ≤x≤x+ws ,y−hs ≤y≤y+hs[D(f −1) (x, y)] + L(f ) (x, y)(1)

where likelihood map is L(f ) , f is the number of frame and size of slope constraint
is denoted as (ws , hs ). We select the highest probability value on the final cumulative DP map. After that, DP searches for the most optimal tracking pathby back-tracking along the highest probability value on each previous likelihood
map. DP is a non-greedy algorithm to estimate the global optimal path in asequence. Due to this, DP-based tracking is robust to occlusion, which degradesa tracking performance of greedy algorithms.
3.3

Synergies by Combining SSD, FCN, and DP

We propose combination of SSD, FCN, and DP as a robust object tracking
method. The proposed method has not only advantage of robustness to appearance variation and occlusion, but also does not need to set a template, change

6Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, Seiichi Uchida

Fig. 2. The tracking path optimization in the proposed method: Each frame is a likelihood map calculated by SSD or FCN and the green arrow suggests a probability valueof a target object at each position. The blue rectangle is a slope constraint to restrict
movement and the red plot is the position where the sum of probability values over
the associated path is highest at the final frame. The orange arrows mean the mostglobally optimal tracking path obtained by back-tracking.

initialization, or change parameters, even when appearance of a target objectchanges.
The proposed method does not need a template for object tracking. For
traditional trackers, a template is necessary and needs to be updated when atarget object changes. However, for the proposed method, it is unnecessary,
under the condition that a category of a target object is trained sufficiently. Since,
the proposed method can deal with appearance variation by learning numerous
features of a target object, it also does not need to modify parameters even if
appearance of a target object is changed. For traditional trackers, identifying
the position of a target object on the first frame is important element to track.
However, the proposed method can obtain the most globally optimal tracking
path by back-tracking over all cumulative DP maps without any identifying theposition.

4

Implementation and Experiments

4.1

Experimental Setup

We used the VOC2012 [31] dataset to train SSD and FCN on 20 categories. The
training dataset has 11,530 images containing 27,450 ROI annotated objects
and 6,929 segmentations. The categories are as follows: person, bird, cat, cow,dog, horse, sheep, airplane, bicycle, boat, bus, car, motorbike, train, bottle, chair,dining table, potted plant, sofa and tv/monitor.
To demonstrate that the proposed method can track a target object with ahigh accuracy, we evaluated the proposed method using sequences that have aGlobally Optimal Object Tracking with SSD and FCN

7

The Proposed Method without Initialization: 0.802
Struck: 0.689
SCM: 0.666
TLD: 0.574
CXT: 0.427
VTD: 0.402

1
0.9

Precision

0.8
0.7
0.6
0.5
0.4
10

20

30

40

50

Location error threshold

Fig. 3. Precision results for single-object sequences: We compared results of the proposed method without initialization and those of the traditional trackers with thatby using single-object sequences. The precision is defined as the percentage of frames
whose estimated position is within location error threshold. The permissible thresholddistance is denoted as location error threshold. The score listed in the legend means theprecision score at a threshold of 20 pixels. The proposed method possesses the higher
performance for all thresholds and has no large deviation from ground-truth.

target object in one of the 20 categories of VOC2012. Since the proposed method
can detect only trained objects on the 20 categories, we selected 12 sequences
for our experiments: CarScale, Coke, Couple, Crossing, David3, Jogging1&2,
MotorRolling, MountainBike, Walking1&2, Woman. It is noteworthy that thosesequences show various difficulties, such as illumination variation, scale variation,
occlusion, fast motion, rotation, and low resolution.
The sequences were classified into two types, single-object sequences1 andmulti-object sequences2 . Since the proposed method is designed to track a single
object without initialization, single-object sequences are appropriate for performance evaluation. The proposed method, however, is still applicable to multiobject sequences by initialization. We therefore conducted two separated experiments, single-object sequences (without initialization) and multi-object sequences (with initialization). Moreover, we conducted two extra experiments.
One is to compare the results using a likelihood map by both SSD and FCN,
single SSD and single FCN. In th, we set N as 10, which is pixel number of1single-object sequences contain CarScale, Coke, Couple, Crossing, David3, MotorRolling, MountainBike, Walking1&2, Woman.
2
multi-object sequences contain CarScale, Coke, Couple, Crossing, David3, Jogging1&2, MotorRolling, MountainBike, Walking1&2, Woman.

8

Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, Seiichi Uchida

The Proposed Method with Initialization: 0.841
Struck: 0.679
SCM: 0.657
TLD: 0.581
CXT: 0.438
VTD: 0.407

1
0.9

Precision

0.8
0.7
0.6
0.5
0.4
10

20

30

40

50

Location error threshold

Fig. 4. Precision results for multi-object sequences: We summarized the all results withinitialization by using multi-object sequences. The precision is defined as the percentage
of frames whose estimated position is within location error threshold. The permissible
threshold distance is denoted as location error threshold. The score listed in the legendmeans the precision score at a threshold of 20 pixels.

the success and failure criteria to switch from SSD to FCN. The other is thecomparison experiment between the proposed method and FCNT [29] which isa tracker using FCN, to demonstrate superiority of the proposed method than
FCNT.
4.2

Evaluation Criterion

We evaluated the proposed method by comparing the precision which is established method in the Visual Tracker Benchmark [11]. The precision is definedas the percentage of frames whose estimated position is within a given threshold from a ground-truth. The distance between the estimated position and themanually labeled ground-truth is calculated by Euclidean distance. To show aperformance efficiently, we conducted one-pass evaluation (OPE) that trackers
run throughout a test sequence only one time and compare the precision of eachof trackers.
4.3

Evaluation Results

Tracking methods can be divided into offine tracking such as the proposed
method and online tracking. However, we compared the proposed method toonline tracking methods in order to show the performance, because there isGlobally Optimal Object Tracking with SSD and FCN

9The Proposed Method (SSD+FCN+DP) without Initialization: 0.802
SSD+DP without Initialization: 0.771
FCN+DP without Initialization: 0.698

1
0.9

Precision

0.8
0.7
0.6
0.5
0.4
10

20

30

40

50

Location error threshold

Fig. 5. Precision results according to how to generate a likelihood map: We compared
the results using likelihood maps by both SSD and FCN, single SSD, and single FCN,
without initialization. The precision is defined as the percentage of frames whose estimated position is within location error threshold. The permissible threshold distance isdenoted as location error threshold. The score listed in the legend means the precision
score at a threshold of 20 pixels. The proposed method which is using likelihood mapsby both SSD and FCN, possesses a higher performance than the others.

no comparable offline tracking methods which are released. We compared the
proposed method to the top five traditional trackers introduced in the Visual
Tracker Benchmark: Structured Output Tracking with Kernels (Struck) [32], asparsity-based tracker (SCM) [33], P-N Learning tracker (TLD) [34], Context
tracker (CXT) [35] and Visual Tracking Decomposition (VTD) [36].
Fig. 3 shows all precision results including the proposed method without
initialization and the traditional trackers with initialization. The score listed
in the legend of Fig. 3 is the precision at a threshold of 20 pixels, since a 20
pixel threshold is the standard threshold for the Visual Tracker Benchmark. As
shown in Fig. 3, we confirmed that the proposed method outperforms than the
traditional trackers, even though the proposed method is not given the initial
position of the object on the first frame. Since DP sets a slope constraint toprohibit tracked position from moving rapidly, the proposed method can tracka target object with small deviation.
As we mentioned in Section 4.1, the proposed method is applicable to multiobject sequences by identifying a initial position of a target object. Fig. 4 shows
all results with initialization. For all thresholds, the proposed method possesses ahigher performance compared to the traditional trackers. Through these results,
when multi-objects even exist on the same frame, the proposed method can track

10

Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, Seiichi Uchida

Traditional trackers

The proposed methodttFig. 6. Example results of the proposed method dealing with appearance variationand occlusion: The center of green and red ’+’ mean the ground-truth and the tracked
position, respectively. Examples of left side are results of the top five traditional trackers
introduced in the Visual Tracker Benchmark. Examples of right side are results of the
proposed method.

a target object distinguishably by initialization. By comparing the results of Fig.
3 and Fig. 4, we also confirm that the precision of the proposed method withinitialization is more accurate than that without initialization.
Fig. 5 shows the precision results using likelihood maps by both SSD and
FCN, a single SSD, and a single FCN, respectively, without initialization. The
proposed method which utilizes both SSD and FCN has a higher performance
than the others at a threshold of 20 pixels. Since the method using single SSD
can not generate likelihood maps for all input images, the results by single SSD
are worse than those by the proposed method. As shown in Fig. 5, the precisionby single FCN ascends rapidly for low thresholds. When a target object is large
enough, the method using single FCN might track the position which is far fromthe center position of a target object, because FCN does not always obtain thehighest probability value which is close to center position of a target object. Dueto this, the precision by single FCN is lower than the others for low thresholds.
Fig. 6 shows examples of the proposed method dealing with appearance variation and occlusion. As shown in Fig. 6, although there are various target objectsof same category in each sequence, the proposed method can track each target
object without template and parameter modification. Also, we confirmed that

Globally Optimal Object Tracking with SSD and FCN

11Fig. 7. Example results of FCNT and the proposed method to track the occluded
target object.

the proposed method can track a occluded target object more stably, in contrast
to the top five traditional trackers, because DP seeks the global optimal pathover all frames. However, when target objects of same category appear with occlusion, such as the final sequence in Fig. 6, the proposed method does not know
which object should track. Since the proposed method utilizes general featuresof same category during object tracking, we should add to specific features of atarget object, when it tracks distinguishably.
We also observed the performance of FCNT [29] using the same sequences.
It could achieve 0.951 and 0.945 precision for single-object and multi-object sequences, respectively, at threshold of 20 pixels. It is, however, almost meaningless
to compare this precision to ours. First of all, we should remember that FCNT
needs a template and ours does not. In addition, FCNT needs initialization andours does not. In fact, we can see that the proposed method without initialization
has no severe degradation from comparison between Fig. 3 and 4. Furthermore,
our DP-based method has theoretical superiority over FCNT at the robustness
to occlusion, as shown in Fig. 7.

5

Conclusion

In this paper, we presented the object tracking method which combines SSD,FCN and DP. We confirmed that the proposed method is robust to appearance
variation and occlusion through several experiments and achieved the highest
accuracy compared to the traditional trackers in the Visual Tracker Benchmark.
In contrast to traditional trackers, the proposed method can track the target object without initialization, modifying parameters, and templates as it synergizes
the combination of SSD, FCN, and DP. However, the proposed method can beextended to tracking with multiple similar objects by using initialization. We
confirmed that using both SSD and FCN is more stable to tracking than single
SSD and single FCN as well.
We expect to use the proposed method in analysis field such as traffic analysis, bio-image analysis, etc. In future, we will connect SSD, FCN with network
flows [37] to track multi-target objects simultaneously. To increase the tracking
accuracy in situation when similar objects of same category appear with occlusion such as the last sequence in Fig. 6, we will apply Flownet [38], in order toutilize information of optical flow.

12

Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, Seiichi Uchida

References
1. Lewis, John P: Fast template matching. In: Vision interface, vol. 95, no. 120123,
pp. 15−19 (1995)
2. Okuma, Kenji and Taleghani, Ali and De Freitas, Nando and Little, James J andLowe, David G: A boosted particle filter: Multitarget detection and tracking. In:
European Conference on Computer Vision, pp. 28−39 (2004)
3. Comaniciu, Dorin and Ramesh, Visvanathan and Meer, Peter: Real-time tracking of
non-rigid objects using mean shift. In: Computer Vision and Pattern Recognition,
vol. 2, pp. 142−149 (2000)
4. Zach, Christopher and Gallup, David and Frahm, Jan-Michael: Fast gain-adaptive
KLT tracking on the GPU. In: Computer Vision and Pattern Recognition, pp. 1−7(2008)
5. He, Wei and Yamashita, Takayoshi and Lu, Hongtao and Lao, Shihong: Surf tracking. In: Computer Vision IEEE 12th International Conference, pp. 1586−1592(2009)
6. Liu, Wei and Anguelov, Dragomir and Erhan, Dumitru and Szegedy, Christian and
Reed, Scott and Fu, Cheng-Yang and Berg, Alexander C: SSD: Single shot multibox
detector. In: European Conference on Computer Vision, pp. 21−37 (2016)7. Long, Jonathan and Shelhamer, Evan and Darrell, Trevor: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431−3440 (2015)
8. Uchida, Seiichi and Sakoe, Hiroaki: A monotonic and continuous two-dimensional
warping based on dynamic programming. In: International Conference on Pattern
Recognition, vol. 1, pp. 521−524 (1998)
9. Geiger, Davi and Gupta, Alok and Costa, Luiz A. and Vlontzos, John: Dynamic
programming for detecting, tracking, and matching deformable contours. In: IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 3, pp.
294−302 (1995)
10. Arnold, James and Shaw, SW and Pasternack, HENRI: Efficient target tracking
using dynamic programming. In: IEEE Transactions on Aerospace and Electronic
Systems, vol. 29, no. 1, pp. 44−56 (1993)
11. Wu, Yi and Lim, Jongwoo and Yang, Ming-Hsuan: Online object tracking: A benchmark. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2411−2418 (2013)
12. Mei, Xue and Ling, Haibin: Robust visual tracking using l1 minimization. In: Computer Vision IEEE 12th International Conference, pp.1436−1443 (2009)
13. Zhang, Tianzhu and Ghanem, Bernard and Liu, Si and Ahuja, Narendra: Robust
visual tracking via multi-task sparse learning. In: Computer Vision and Pattern
Recognition (CVPR), pp. 2042−2049 (2012)
14. Han, Bohyung and Comaniciu, Dorin and Zhu, Ying and Davis, Larry S: Sequential
kernel density approximation and its application to real-time visual tracking. In:
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 7, pp.
1186−1197 (2008)
15. Jepson, Allan D and Fleet, David J and El-Maraghi, Thomas F: Robust online
appearance models for visual tracking. In: IEEE transactions on pattern analysis
and machine intelligence, vol. 25, no. 10, pp. 1296−1311 (2003)
16. Ross, David A and Lim, Jongwoo and Lin, Ruei-Sung and Yang, Ming-Hsuan: Incremental learning for robust visual tracking. In: International Journal of Computer
Vision, vol. 77, no. 1, pp. 125−141 (2008)

Globally Optimal Object Tracking with SSD and FCN

13

17. Kalal, Zdenek and Mikolajczyk, Krystian and Matas, Jiri: Tracking-learning detection. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
34, no. 7, pp. 1409−1422 (2012)
18. Grabner, Helmut and Grabner, Michael and Bischof, Horst: Real-time tracking via
on-line boosting. In: BMVC, vol. 1, no. 5, pp. 6 (2006)
19. Grabner, Helmut and Leistner, Christian and Bischof, Horst: Semi-supervised online boosting for robust tracking. In: ECCV, pp. 234−247 (2008)
20. Son, Jeany and Jung, Ilchae and Park, Kayoung and Han, Bohyung: Tracking-by
segmentation with online gradient boosting decision tree. In: Proceedings of theIEEE International Conference on Computer Vision, pp. 3056−3064 (2015)
21. Lawrence, Steve and Giles, C Lee and Tsoi, Ah Chung and Back, Andrew D: Face
recognition: A convolutional neural-network approach. In: IEEE transactions onneural networks, vol. 8, no. 1, pp. 98−113 (1997)
22. Ciresan, Dan Claudiu and Meier, Ueli and Gambardella, Luca Maria and Schmidhuber, Jurgen: Convolutional neural network committees for handwritten characterclassification. In: Document Analysis and Recognition (ICDAR), pp. 1135−1139(2011)
23. Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E: Imagenet classification with deep convolutional neural networks. In: Advances in neural information
processing systems, pp. 1097−1105 (2012)
24. Sainath, Tara N and Mohamed, Abdel-rahman and Kingsbury, Brian and Ramabhadran, Bhuvana: Deep convolutional neural networks for LVCSR. In: Acoustics,
speech and signal processing (ICASSP), pp. 8614−8618 (2013)
25. Fan, Jialue and Xu, Wei and Wu, Ying and Gong, Yihong: Human tracking using
convolutional neural networks. In: IEEE Transactions on Neural Networks, vol. 21,
no. 10, pp. 1610−1623 (2010)
26. Maung, Tin Hninn Hninn: Real-time hand tracking and gesture recognition systemusing neural networks. In: World Academy of Science, Engineering and Technology,
vol. 50, pp. 466−470 (2009)
27. Torricelli, Diego and Conforto, Silvia and Schmid, Maurizio and DfAlessio, Tommaso: A neural-based remote eye gaze tracker under natural head motion. In: Computer methods and programs in biomedicine, vol. 92, no. 1, pp. 66−78 (2008)
28. Li, Hanxi and Li, Yi and Porikli, Fatih: Robust Online Visual Tracking with aSingle Convolutional Neural Network. In: Asian Conference on Computer Vision,
pp. 194−209 (2014)
29. Wang, Lijun and Ouyang, Wanli and Wang, Xiaogang and Lu, Huchuan: Visualtracking with fully convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3119−3127 (2015)30. Simonyan, Karen and Zisserman, Andrew: Very deep convolutional networks forlarge-scale image recognition. In: CoRR, vol. abs/1409.1556 (2014)31. Everingham, Mark and Eslami, SM Ali and Van Gool, Luc and Williams, Christopher KI and Winn, John and Zisserman, Andrew: The pascal visual object classes
challenge: A retrospective. In: International Journal of Computer Vision, vol. 111,
no. 1, pp. 98−136 (2015)32. Hare, Sam and Golodetz, Stuart and Saffari, Amir and Vineet, Vibhav and Cheng,
Ming-Ming and Hicks, Stephen L and Torr, Philip HS: Structured output tracking
with kernels. In: IEEE transactions on pattern analysis and machine intelligence,
vol. 38, no. 10, pp. 2096−2109 (2016)
33. Zhong, Wei and Lu, Huchuan and Yang, Ming-Hsuan: Robust object tracking
via sparsity-based collaborative model. In: Proceedings of the IEEE conference on
computer vision and pattern recognition, pp. 1838−1845 (2012)

14

Jinho Lee, Brian Kenji Iwana, Shouta Ide, Hideaki Hayashi, Seiichi Uchida

34. Kalal, Zdenek and Matas, Jiri and Mikolajczyk, Krystian: Pn learning: Bootstrapping binary classifiers by structural constraints. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 49−56 (2010)
35. Dinh, Thang Ba and Vo, Nam and Medioni, Gérard: Context tracker: Exploring
supporters and distracters in unconstrained environments. In: Proceedings of theIEEE conference on computer vision and pattern recognition, pp. 1177−1184 (2011)
36. Kwon, Junseok and Lee, Kyoung Mu: Visual tracking decomposition. In: Computer
Vision and Pattern Recognition (CVPR), pp. 1269−1276 (2010)
37. Zhang, Li and Li, Yuan and Nevatia, Ramakant: Global data association for multiobject tracking using network flows. In: Computer Vision and Pattern Recognition,
pp. 1−8 (2008)
38. Dosovitskiy, Alexey and Fischer, Philipp and Ilg, Eddy and Hausser, Philip andHazirbas, Caner and Golkov, Vladimir and van der Smagt, Patrick and Cremers,
Daniel and Brox, Thomas: Flownet: Learning optical flow with convolutional networks. In: IEEE International Conference on Computer Vision, pp. 2758−2766(2015)