# Part-Based Visual Tracking via Online Weighted P-N Learning

Authors: Heng Fan; Jinhai Xiang; Jun Xu; Honghong Liao
Journal: The Scientific World Journal
 (2014)

---

## Abstract

We propose a novel part-based tracking algorithm using online weighted P-N learning. An online weighted P-N learning method is implemented via considering the weight of samples during classification, which improves the performance of classifier. We apply weighted P-N learning to track a part-based target model instead of whole target. In doing so, object is segmented into fragments and parts of them are selected as local feature blocks (LFBs). Then, the weighted P-N learning is employed to train classifier for each local feature block (LFB).  Each LFB is tracked through the corresponding classifier, respectively. According to the tracking results of LFBs, object can be then located. During tracking process, to solve the issues of occlusion or pose change, we use a substitute strategy to dynamically update the set of LFB, which makes our tracker robust. Experimental results demonstrate that the proposed method outperforms the state-of-the-art trackers.


---

## Body

## 1. Introduction

Object tracking is one of the most important components of many applications in computer vision, such as human computer interactions, surveillance, and robotics. However, robust visual tracking is still a challenging problem, which is affected by partial or full occlusion, illumination, scale, and poses variation [1]. The key for the object tracking is to construct an effective appearance model. Many tracking algorithms have been proposed recently, but designing a robust appearance model is still a major challenge, which is affected by both extrinsic (e.g., illumination variation, background clutter, and partial or full occlusion) and intrinsic (e.g., scale and pose variation) factors. In order to handle these problems, a wide range of appearance models based on different visual representations and statistical modeling techniques have been presented by researchers. In general, these appearance models can be categorized into two types: appearance model based on visual representation, such as global-based representation [2–6] and local-based representation [7–11]; appearance model based on statistical modeling, such as generative model [12–16] and discriminative model [5, 6, 17–20].In this paper, we propose a part-based visual tracking algorithm with online weighted P-N learning. Weighted P-N learning is first proposed by assigning weights (property weight and classification weight) to each sample in training sample set, which can decrease false classification and improve the discriminative power of classifier. Then, we segment object into fragments and select parts of them as local feature blocks to represent the object. Finally, we train classifier for each LFB with weighted P-N learning to obtain the corresponding classifier, respectively, and track each LFB independently within the framework of Lucas-Kanade optical flow [21]. During tracking process, a real-time valid detection method is used for each LFB. If certain LFB is invalid, we use a replacing strategy to update the local feature block set, which can ensure successful tracking.Contributions. The contributions of this paper include the following.(i)
A part-based visual tracking algorithm with online weighted P-N learning is proposed in this work. Object is represented by LFBs and tracked. When occlusion or distortion happens, a strategy is adopted to replace invalid LFB and keep the new LFB set effective.
(ii)
We define the weights (property weight and classification weight) for each sample in training process of P-N learning.
(iii)
An online weighted P-N learning is presented by assigning weight to each sample in training sample set, which can improve discriminative power of classifier by decreasing classification errors and increasing the accuracy of tracker.The rest of the paper is organized as follows. Section2 reviews the related work of this paper. Section 3 introduces weighted P-N learning. Proposed tracking method is presented in detail in Section 4. Experimental results are shown in Section 5. Section 6 concludes the whole paper.


## 2. Related Work

Recently, many trackers based on local feature representation have been proposed. Adam et al. [8] present a fragment-based tracking approach, and further, Wang et al. [22] embed the fragment-based method into mean shift tracking framework. This tracking method estimates the target based on voting map of each part via comparing its histogram with the template’s. Nevertheless, static template with equal importance being assigned to each fragment obviously lowers the performance of tracker. In order to overcome the shortcomings, Jia et al. [7] propose a fragment-based tracking method using online multiple kernel learning (MKL). All the patches are assigned to different weights based on the importance learned by MKL. However, this strategy may still cause drifting problem. Occlusion especially, which makes part patches invalid, leads to errors in computing voting map, even tracking failure. Wang et al. [10] introduce a tracking method based on superpixel. It only computes the probabilities of superpixels belonging to target, which is prone to drift away in color-similar background and whose tracking results will shrink to the unoccluded part of object when occlusion happens.Another type of tracking method is based on discriminative appearance model. Tang et al. [23] present a tracking method based on semisupervised support vector machines. This tracker employs a small number of labeled samples for semi-supervised learning and develops a classifier to mark the unlabeled data. Babenko et al. [5] propose a multiple instance learning (MIL) method for visual tracking. This approach solves the problem of slight inaccuracies in the tracker leading to incorrectly labeled training samples and can alleviate drift problem to some extent. However, the MIL tracker might detect positives, which are less important because they do not consider the importance of sample in learning process. Further, Zhang and Song [20] suggest a weighted multiple instance learning (WMIL) tracking method. It assigns weight to each sample based on the corresponding importance. This approach improves the robustness of tracker. Kalal et al. [17] propose a method called P-N learning, which learns from positive samples and negative samples, to construct a classifier. In the meanwhile, the discriminative properties of classifier are improved by two categories of constrains that are termed P-constrains and N-constrains. However, false classification in P-N learning degrades the classifier in some degree.Another work similar to ours is [9], which utilizes blocks to represent the object. However, the blocks are easily invalided when target appearance changes, which undermines its robustness to nonrigid distortion or occlusion. In our work, we employ LFBs to represent target and use a dynamically updating mechanism to update the local feature block set, which guarantees each LFB in the set is valid when occlusion or deformation occurs. Hence, our tracker is more robust and effective.


## 3. Weighted P-N Learning

P-N learning is a semisupervised online learning algorithm proposed by Kalal et al. [17, 24–26]. Let x be a sample in feature space X, and let y be a label in label space Y
=
{
1
,
-
1
}. A set of samples X and corresponding set of labels Y are defined as (
X
,
Y
), which is termed a labeled set. The aim of P-N learning is to develop a binary classifier f
:
X
→
Y based on a priori labeled set (
X
l
,
Y
l
) and improve its discriminative performance by unlabeled data X
u. The flowchart of P-N learning approach in [17] is shown in Figure 1.Figure 1
A flowchart in [17] to explain P-N learning. P-N learning algorithm initially develops a classifier from prior knowledge and then iterates over: (1) classify the unlabeled data and label it; (2) reclassify the samples within constrains and label them; (3) expand the training sample set; (4) retrain the classifier.


### 3.1. Classifier Bootstrapping

The binary classifierf is a function parameterized by θ. Similar to supervised learning, P-N learning is to estimate parameter θ via training sample set (
X
t
,
Y
t
). Nevertheless, it is worth noticing that the training set is iteratively expanded through adding samples, which is screened by constraints from unlabeled data. Initially, classifier and its parameter θ
0 are obtained by training labeled samples. Then, the process proceeds iteratively. In iteration k, all the unlabeled samples are marked by classifier in iteration  k
-
1; namely,
(1)
y
u
k
=
f
(
x
u
∣
θ
k
-
1
)
,
x
u
∈
X
u
.
Then, the constraints are utilized to revise the classification results and add the corrected labels to training set. Iteration k ends with retraining classifier by using the renewed training sample set.During training process, the classifier may identify the unlabeled data with wrong labels. Any sample may be screened many times with constraints and hence can be represented mistakenly in the training set repeatedly. Obviously, it can significantly degrade discriminative performance of classifier and therefore lower the accuracy of tracking. In order to further improve the accuracy and robustness of the classifier, we propose a weighted P-N learning method by assigning weight to each sample in training set. Samplej in training set has two categories of weight which are termed P-weight W
j
+ and N-negative W
j
-. P-weight represents the probability of being a positive sample, and N-weight represents the probability of being a negative sample. In iteration k, sample j from training set is represented as positive sample for C
j
+ times and as negative sample for C
j
- times. The positive weight W
1
j
+ and negative weight W
1
j
- are determined by the following formulation:
(2)
W
1
j
+
=
C
j
+
C
j
+
+
C
j
-
,
W
1
j
-
=
C
j
-
C
j
+
+
C
j
-
.
Besides, the probability of sample j being positive or negative in training set obtained by classifier is defined as classification weights W
2
j
+ and W
2
j
- (in Section 3.3). The P-weight and N-weight of sample j can be then obtained by the following formulation:
(3)
W
j
+
=
W
1
j
+
+
W
2
j
+
,
W
j
-
=
W
1
j
-
+
W
2
j
-
.
At last, sample j is determined to be either positive or negative via the following formulation:
(4)
W
j
+
W
j
-
≥
1
positive sample,
W
j
+
W
j
-
<
1
negative sample.Figure2 demonstrates the tracking results with weighted P-N learning. In Figure 2, the left and middle images are the ground truth and tracking results with weighted P-N learning, and the right images are tracking results based on P-N learning.(a) Ground truth. (b) Tracking results with weighted P-N learning. (c) Tracking results based on P-N learning.
(a)
(b)
(c)


### 3.2. Constraints

In P-N learning, a constraint can be arbitrary function, especially two categories of constraints which we term P and N. P-constrains recognize samples which are labeled negative by the classifier, yet constraints need a positive label. P-constraints addn
+
(
k
) samples to the training set in iteration k. Similarly, N-constraints are employed to identify samples classified as positive but constraints require negative label. In iteration k, N-constraints insert n
-
(
k
) samples to training set.In iterationk, the error of a classifier is represented by a number of false positives α
(
k
) and a number of false negatives β
(
k
). Let n
c
+
(
k
) be the number of samples for which the label is correctly changed to positive in iteration k by P-constraints, and n
f
+
(
k
) is then the number of samples for which the label is incorrectly changed to positive in iteration k. Hence, P-constraints change n
+
(
k
)
=
n
c
+
(
k
)
+
n
f
+
(
k
) samples to positive. Similarly, N-constraints change n
-
(
k
)
=
n
c
-
(
k
)
+
n
f
-
(
k
) samples to negative, where n
c
-
(
k
) and n
f
-
(
k
) are correct and false assignments. The errors of classifier can be represented as the following formulations:
(5)
α
(
k
+
1
)
=
α
(
k
)
-
n
c
-
(
k
)
+
n
f
+
(
k
)
,
(6)
β
(
k
+
1
)
=
β
(
k
)
-
n
c
+
(
k
)
+
n
f
-
(
k
)
.
Equation (5) demonstrates that false positives α
(
k
) decrease if n
c
-
(
k
)
>
n
f
+
(
k
). In the similar way, false negatives β
(
k
) decrease if n
c
+
(
k
)
>
n
f
-
(
k
). To analyze the convergence of learning process, a model needs to be developed that relates the performance of P-N constraints to n
c
+
(
k
), n
c
-
(
k
), n
f
+
(
k
), and n
f
-
(
k
).The performance of P-N constraints is represented by four indexes, P-precisionP
+, P-recall R
+, N-precision P
-, and N-recall R
-, determined by the following formulation:
(7)
P
(
k
)
+
=
n
c
+
(
k
)
n
c
+
(
k
)
+
n
f
+
(
k
)
,
R
(
k
)
+
=
n
c
+
(
k
)
β
(
k
)
,
P
(
k
)
-
=
n
c
-
(
k
)
n
c
-
(
k
)
+
n
f
-
(
k
)
,
R
(
k
)
-
=
n
c
-
(
k
)
α
(
k
)
.
According to formulation (7), it is easy to get
(8)
n
c
+
(
k
)
=
R
(
k
)
+
·
β
(
k
)
,
n
f
+
(
k
)
=
1
-
P
(
k
)
+
P
(
k
)
+
·
R
(
k
)
+
·
β
(
k
)
,
n
c
-
(
k
)
=
R
(
k
)
-
·
α
(
k
)
,
n
f
-
(
k
)
=
1
-
P
(
k
)
-
P
(
k
)
-
·
R
(
k
)
-
·
α
(
k
)
.
By combining formulations (5), (6), and (8), we can obtain new formulations:
(9)
α
(
k
+
1
)
=
(
1
-
R
(
k
)
-
)
·
α
(
k
)
+
1
-
P
(
k
)
+
P
(
k
)
+
·
R
(
k
)
+
·
β
(
k
)
,
β
(
k
+
1
)
=
1
-
P
(
k
)
-
P
(
k
)
-
·
R
(
k
)
-
·
α
(
k
)
+
(
1
-
R
(
k
)
+
)
·
β
(
k
)
.After defining state vectorx
→
(
k
)
=
[
α
(
k
)
β
(
k
)
]
T and transition matrix M as the following,
(10)
M
=
[
1
-
R
(
k
)
-
1
-
P
(
k
)
+
P
(
k
)
+
·
R
(
k
)
+
1
-
P
(
k
)
-
P
(
k
)
-
·
R
(
k
)
-
1
-
R
(
k
)
+
]
,
hence formulation (9) can be rewritten as the following formulation:
(11)
x
→
(
k
+
1
)
=
M
x
→
(
k
)
.
According to [27], formulation (11) is a recursive equation that is related to a discrete dynamical system. Based on the theory of dynamical systems, the state vector x
→ converges to zero if eigenvalues λ
1 and λ
2 of the transition matrix M meet the condition λ
1
<
1 and λ
2
<
1. As pointed in [17], the performance of classifier will be improved constantly, only if the two eigenvalues of transition matrix M are smaller than one.


### 3.3. Object Detecting

In previous subsections, we illustrate the weighted P-N learning method. In this subsection, a classifier will be developed to detect the object. Scanning window strategy is utilized to detect the object in [17]. Similarly, we use this method to detect the object.In this paper, the randomized forest classifier [28] is adopted. For each input subwindow, classifier consists of N ferns. Each fern i computes the input patch resulting in feature vector x
i, which is used to obtain posterior probability P
(
y
=
1
∣
x
i
). The following formulation is defined to discriminate input patch:
(12)
P
avg
>
λ
object
,
P
avg
≤
λ
background
,
where P
avg
=
∑
i
=
1
N
P
(
y
=
1
∣
x
i
) denotes the average of all posteriors and λ is the threshold which is set to 0.4 in all experiments. The detection process can be illustrated in Figure 3. Actually, W
2
j
+
=
P
avg and W
2
j
-
=
1
-
P
avg. Feature vector is represented by 2-bit Binary Patterns [25] because of their invariance to illumination and efficient multiscale implementation using integral image. In fact, the posteriors P
(
y
=
1
∣
x
i
) represent the parameter θ of the classifier and are estimated incrementally through the entire learning process. Each leafnode of fern records the number of positive p and negative n samples changed into it during iteration. The posteriors are then estimated by the following formulation:
(13)
P
(
y
=
1
∣
x
i
)
=
p
p
+
n
leaf is not empty,
P
(
y
=
1
∣
x
i
)
=
0
leaf is empty.Figure 3
Object detection based on scanning window strategy and randomized forest classifier. The setting of the detector is as follows: 10,000 windows are scanned, 10 ferns per window.The classifier is initialized in first frame, and posteriors are initialized to zero and renewed by 500 positive samples produced by affine warping of the selected patch [1]. The classifier is then evaluated on all the patches. In this paper, detections far from the selected patch represent the negative samples and update the posteriors.


## 3.1. Classifier Bootstrapping

The binary classifierf is a function parameterized by θ. Similar to supervised learning, P-N learning is to estimate parameter θ via training sample set (
X
t
,
Y
t
). Nevertheless, it is worth noticing that the training set is iteratively expanded through adding samples, which is screened by constraints from unlabeled data. Initially, classifier and its parameter θ
0 are obtained by training labeled samples. Then, the process proceeds iteratively. In iteration k, all the unlabeled samples are marked by classifier in iteration  k
-
1; namely,
(1)
y
u
k
=
f
(
x
u
∣
θ
k
-
1
)
,
x
u
∈
X
u
.
Then, the constraints are utilized to revise the classification results and add the corrected labels to training set. Iteration k ends with retraining classifier by using the renewed training sample set.During training process, the classifier may identify the unlabeled data with wrong labels. Any sample may be screened many times with constraints and hence can be represented mistakenly in the training set repeatedly. Obviously, it can significantly degrade discriminative performance of classifier and therefore lower the accuracy of tracking. In order to further improve the accuracy and robustness of the classifier, we propose a weighted P-N learning method by assigning weight to each sample in training set. Samplej in training set has two categories of weight which are termed P-weight W
j
+ and N-negative W
j
-. P-weight represents the probability of being a positive sample, and N-weight represents the probability of being a negative sample. In iteration k, sample j from training set is represented as positive sample for C
j
+ times and as negative sample for C
j
- times. The positive weight W
1
j
+ and negative weight W
1
j
- are determined by the following formulation:
(2)
W
1
j
+
=
C
j
+
C
j
+
+
C
j
-
,
W
1
j
-
=
C
j
-
C
j
+
+
C
j
-
.
Besides, the probability of sample j being positive or negative in training set obtained by classifier is defined as classification weights W
2
j
+ and W
2
j
- (in Section 3.3). The P-weight and N-weight of sample j can be then obtained by the following formulation:
(3)
W
j
+
=
W
1
j
+
+
W
2
j
+
,
W
j
-
=
W
1
j
-
+
W
2
j
-
.
At last, sample j is determined to be either positive or negative via the following formulation:
(4)
W
j
+
W
j
-
≥
1
positive sample,
W
j
+
W
j
-
<
1
negative sample.Figure2 demonstrates the tracking results with weighted P-N learning. In Figure 2, the left and middle images are the ground truth and tracking results with weighted P-N learning, and the right images are tracking results based on P-N learning.(a) Ground truth. (b) Tracking results with weighted P-N learning. (c) Tracking results based on P-N learning.
(a)
(b)
(c)


## 3.2. Constraints

In P-N learning, a constraint can be arbitrary function, especially two categories of constraints which we term P and N. P-constrains recognize samples which are labeled negative by the classifier, yet constraints need a positive label. P-constraints addn
+
(
k
) samples to the training set in iteration k. Similarly, N-constraints are employed to identify samples classified as positive but constraints require negative label. In iteration k, N-constraints insert n
-
(
k
) samples to training set.In iterationk, the error of a classifier is represented by a number of false positives α
(
k
) and a number of false negatives β
(
k
). Let n
c
+
(
k
) be the number of samples for which the label is correctly changed to positive in iteration k by P-constraints, and n
f
+
(
k
) is then the number of samples for which the label is incorrectly changed to positive in iteration k. Hence, P-constraints change n
+
(
k
)
=
n
c
+
(
k
)
+
n
f
+
(
k
) samples to positive. Similarly, N-constraints change n
-
(
k
)
=
n
c
-
(
k
)
+
n
f
-
(
k
) samples to negative, where n
c
-
(
k
) and n
f
-
(
k
) are correct and false assignments. The errors of classifier can be represented as the following formulations:
(5)
α
(
k
+
1
)
=
α
(
k
)
-
n
c
-
(
k
)
+
n
f
+
(
k
)
,
(6)
β
(
k
+
1
)
=
β
(
k
)
-
n
c
+
(
k
)
+
n
f
-
(
k
)
.
Equation (5) demonstrates that false positives α
(
k
) decrease if n
c
-
(
k
)
>
n
f
+
(
k
). In the similar way, false negatives β
(
k
) decrease if n
c
+
(
k
)
>
n
f
-
(
k
). To analyze the convergence of learning process, a model needs to be developed that relates the performance of P-N constraints to n
c
+
(
k
), n
c
-
(
k
), n
f
+
(
k
), and n
f
-
(
k
).The performance of P-N constraints is represented by four indexes, P-precisionP
+, P-recall R
+, N-precision P
-, and N-recall R
-, determined by the following formulation:
(7)
P
(
k
)
+
=
n
c
+
(
k
)
n
c
+
(
k
)
+
n
f
+
(
k
)
,
R
(
k
)
+
=
n
c
+
(
k
)
β
(
k
)
,
P
(
k
)
-
=
n
c
-
(
k
)
n
c
-
(
k
)
+
n
f
-
(
k
)
,
R
(
k
)
-
=
n
c
-
(
k
)
α
(
k
)
.
According to formulation (7), it is easy to get
(8)
n
c
+
(
k
)
=
R
(
k
)
+
·
β
(
k
)
,
n
f
+
(
k
)
=
1
-
P
(
k
)
+
P
(
k
)
+
·
R
(
k
)
+
·
β
(
k
)
,
n
c
-
(
k
)
=
R
(
k
)
-
·
α
(
k
)
,
n
f
-
(
k
)
=
1
-
P
(
k
)
-
P
(
k
)
-
·
R
(
k
)
-
·
α
(
k
)
.
By combining formulations (5), (6), and (8), we can obtain new formulations:
(9)
α
(
k
+
1
)
=
(
1
-
R
(
k
)
-
)
·
α
(
k
)
+
1
-
P
(
k
)
+
P
(
k
)
+
·
R
(
k
)
+
·
β
(
k
)
,
β
(
k
+
1
)
=
1
-
P
(
k
)
-
P
(
k
)
-
·
R
(
k
)
-
·
α
(
k
)
+
(
1
-
R
(
k
)
+
)
·
β
(
k
)
.After defining state vectorx
→
(
k
)
=
[
α
(
k
)
β
(
k
)
]
T and transition matrix M as the following,
(10)
M
=
[
1
-
R
(
k
)
-
1
-
P
(
k
)
+
P
(
k
)
+
·
R
(
k
)
+
1
-
P
(
k
)
-
P
(
k
)
-
·
R
(
k
)
-
1
-
R
(
k
)
+
]
,
hence formulation (9) can be rewritten as the following formulation:
(11)
x
→
(
k
+
1
)
=
M
x
→
(
k
)
.
According to [27], formulation (11) is a recursive equation that is related to a discrete dynamical system. Based on the theory of dynamical systems, the state vector x
→ converges to zero if eigenvalues λ
1 and λ
2 of the transition matrix M meet the condition λ
1
<
1 and λ
2
<
1. As pointed in [17], the performance of classifier will be improved constantly, only if the two eigenvalues of transition matrix M are smaller than one.


## 3.3. Object Detecting

In previous subsections, we illustrate the weighted P-N learning method. In this subsection, a classifier will be developed to detect the object. Scanning window strategy is utilized to detect the object in [17]. Similarly, we use this method to detect the object.In this paper, the randomized forest classifier [28] is adopted. For each input subwindow, classifier consists of N ferns. Each fern i computes the input patch resulting in feature vector x
i, which is used to obtain posterior probability P
(
y
=
1
∣
x
i
). The following formulation is defined to discriminate input patch:
(12)
P
avg
>
λ
object
,
P
avg
≤
λ
background
,
where P
avg
=
∑
i
=
1
N
P
(
y
=
1
∣
x
i
) denotes the average of all posteriors and λ is the threshold which is set to 0.4 in all experiments. The detection process can be illustrated in Figure 3. Actually, W
2
j
+
=
P
avg and W
2
j
-
=
1
-
P
avg. Feature vector is represented by 2-bit Binary Patterns [25] because of their invariance to illumination and efficient multiscale implementation using integral image. In fact, the posteriors P
(
y
=
1
∣
x
i
) represent the parameter θ of the classifier and are estimated incrementally through the entire learning process. Each leafnode of fern records the number of positive p and negative n samples changed into it during iteration. The posteriors are then estimated by the following formulation:
(13)
P
(
y
=
1
∣
x
i
)
=
p
p
+
n
leaf is not empty,
P
(
y
=
1
∣
x
i
)
=
0
leaf is empty.Figure 3
Object detection based on scanning window strategy and randomized forest classifier. The setting of the detector is as follows: 10,000 windows are scanned, 10 ferns per window.The classifier is initialized in first frame, and posteriors are initialized to zero and renewed by 500 positive samples produced by affine warping of the selected patch [1]. The classifier is then evaluated on all the patches. In this paper, detections far from the selected patch represent the negative samples and update the posteriors.


## 4. Tracking

In this paper, the object is represented by independent local feature blocks. The tracking task is then transformed into tracking each local feature block. We train classifier for each LFB with online weighted P-N learning, respectively, and then track each LFB independently within the framework of LK optical flow [21]. During tracking procedure, a real-time valid detection algorithm is utilized for each LFB. If certain LFB is invalid, it will be replaced with an unused block, which makes our tracker robust. Figure 4 illustrates the principle of tracking.Figure 4
Illustration of tracking process. Part (I): segmentation of object in initial frame (first frame). Object is divided into six blocks. Part (II): tracking object without occlusion. Green block and blue block are selected as LFBs in image (b). Green and blue dotted line rectangles in image (c) are the tracking results of LFBs, and the red dotted line rectangle in image (d) represents the location of object estimated by the tracking results of LFBs. Part (III): tracking object with occlusion yet valid LFBs. Essentially, this process is similar to part (II). Occlusion does not affect the tracking. Part (IV): tracking object with occlusion with invalid LFB. In this part, green LFB is occluded and therefore invalid. We replace it with another new valid LFB, namely, the pink LFB, as shown in image (h). The object is then located in image (j) by tracking results of blue LFB and pink LFB in image (i).


### 4.1. Set of Local Feature Blocks

Object is represented by LFBs, and thereby, object needs to be segmented into fragments. For simplicity, uniform segmentation is adopted in this paper as shown in Figure5.Figure 5
Uniform segmentation of object. Image (a) is object and image (b) is fragments.After segmentation, we select part blocks as LFBs. Assume object is divided intoK blocks; then we can obtain a candidate set of LFB set C
B
=
{
b
1
,
b
2
,
…
,
b
K
} with K candidate local feature blocks. For candidate LFB b
i, we compute its 2-bit Binary Patterns feature vector x
i. Then, scanning window method is used to compute the similar likelihood between feature vectors of input patch f
i and b
i, and the similarity is represented as L
i
j. M
L
i
=
max
⁡
(
L
i
j
) represents the highest similarity between b
i and all input patches. Finally, local feature block set S
B
=
{
s
b
1
,
s
b
2
,
…
,
s
b
M
} consists of M
(
M
<
K
) candidate local feature blocks, with smaller M
L
i.


### 4.2. Representations

In framet, B
t
=
(
C
t
,
R
t
,
W
t
,
H
t
) represents the object, where C
t and R
t are coordinates of center position and W
t and H
t are the sizes of object. s
b
k
t
=
(
c
k
t
,
r
k
t
,
w
k
t
,
h
k
t
) is the k
th LFB, where c
k
t and r
k
t are coordinates of center location and w
k
t and h
k
t are the sizes of local feature block. z
k
t
=
(
o
c
k
t
,
o
r
k
t
,
r
w
k
t
,
r
h
k
t
) represents the offset of the k
th LFB relative to object, where o
c
k
t and o
r
k
t are the offset of center coordinates and r
w
k
t and r
h
k
t are the rations of sizes between object and the k
th LFB. z
k
t can be determined by the following formulation:
(14)
o
c
k
t
=
C
t
-
c
k
t
,
o
r
k
t
=
R
t
-
r
k
t
,
r
w
k
t
=
W
t
w
k
t
,
r
h
k
t
=
H
t
h
k
t
.


### 4.3. Object Tracking

The tracked target is determined in initial frame (first frame) and segmented into fragments according to Section4.1. Then, we select local feature blocks and compute B
1, s
b
k
1, and z
k
1.Assume current frame ist. Each LFB is corresponded with a classifier via weighted P-N learning, and then we track each LFB. For the k
th LFB, s
b
k
t
+
1
=
(
c
k
t
+
1
,
r
k
t
+
1
,
w
k
t
+
1
,
h
k
t
+
1
) is used to represent its tracking result in frame t
+
1. By combining tracking results of each LFB and its offset z
k
t, we can obtain the corresponding object B
k
t
+
1
=
(
C
k
t
+
1
,
R
k
t
+
1
,
W
k
t
+
1
,
H
k
t
+
1
) via formulation the following formulation:
(15)
C
k
t
+
1
=
o
c
k
t
+
c
k
t
+
1
,
R
k
t
+
1
=
o
r
k
t
+
r
k
t
+
1
,
W
k
t
+
1
=
r
w
k
t
·
w
k
t
+
1
,
H
k
t
+
1
=
r
h
k
t
·
h
k
t
+
1
.
Each LFB determines a related candidate object. Object finally can be located by the following formulation:
(16)
B
t
+
1
=
1
M
·
∑
i
=
1
M
‍
B
i
t
+
1
,
where M is the number of LFBs. The entire process can be explained by Figure 6. An adjustment is needed for offset of each LFB relative to object. We divide new object region B
t
+
1 into fragments and compute the new offset of each LFB with prior LFBs based on formulation (14). After this, classifier of each LFB needs to be retrained via weighted P-N learning.Figure 6
The process of locating object. The green, pink, and blue patches are the local feature blockss
b
1
t
+
1, s
b
2
t
+
1, and s
b
3
t
+
1, and the green, blue, and pink dotted line rectangles are the corresponding tracking results B
1
t
+
1, B
2
t
+
1, and B
3
t
+
1. The red solid line rectangle represents the final tracking result B
t
+
1, which is located by tracking results of local feature blocks.Representation by LFBs has significant advantages. Firstly, compared with entire object, local feature block is more prone to recognition in background, and this guarantees the accuracy and stability of tracking. Besides, object is located by averaging all the tracking results of local feature blocks, which decreases tracking errors by counteracting positive and negative errors; therefore, the robustness of proposed algorithm is improved.


### 4.4. Updating Set of Local Feature Blocks

During the tracking process, update is essentially adaptive to complex environment variation. In this paper, the learning procedure is online, and then the main problem is how to handle the situation of local feature blocks being invalid. A strategy of replacing is adopted to solve this problem. During the tracking procedure, we make a real-time valid detection (in Section3.3) for all local feature blocks. If P
avg
≤
λ, local feature block is invalid. When certain local feature block is invalid, it will be replaced with an appropriate block, which is selected from the outside of the LFB set.LetU
B be the unused block set, and U
B
=
{
u
b
1
,
u
b
2
,
…
,
u
b
P
}
,
P
=
K
-
M
,
U
B
⋃
S
B
=
C
B. In current frame t, LFB s
b
k
t from S
B is invalid and needs to be replaced. We first segment object into blocks and obtain U
B. For block j from U
B, we compute similar likelihood l
j
t
=
sim
(
x
j
t
,
x
j
q
), where sim is function of computing similar likelihood, x
j
t is feature vector of block j in frame t, and x
j
q is feature vector of block j before it is used the last time in frame q (if block j is never used, q equals one). u
b
τ is used to replace s
b
k
t via the following formulation:
(17)
u
b
τ
=
arg
max
⁡
⁡
l
j
t
(
u
b
j
)
.
The whole update process can be illustrated as shown in Figure 7.The process of updating. The object is located by LFBs in (a) without occlusion. When occlusion occurs, yet each LFB is valid, the object can be tracked by LFBs exactly, as shown in (b). If occlusion happens and certain LFB is invalid, then it will be replaced and target still can be successfully located, as shown in (c), (d), (e), and (f).
(a)
(b)
(c)
(d)
(e)
(f)So far, we have introduced the overall procedure of the proposed tracking algorithm as shown in Algorithm1.Algorithm 1:Tracking based on proposed method.
Initialization:
(1) Segment object intoN blocks and obtain C
B
=
{
c
b
i
}
i
=
1
N;
(2) Select the set of LFBsS
B
=
{
s
b
k
}
k
=
1
M
(
M
<
N
);
(3) Compute the set of offset in first frame{
z
k
1
}
k
=
1
M;
(4) Generate positive samples(
X
k
+
,
Y
k
+
) and negative
samples(
X
k
-
,
Y
k
-
) for the k
t
h LFB;
(5) Train classifierf
k for the k
th LFB with data (
X
,
Y
) via
weighted P-N learning, whereX
=
{
X
k
+
,
X
k
-
},
andY
=
{
Y
k
+
,
Y
k
-
};
Object Tracking:
(6)for
t
=
2 to the end of the sequence
do
(7)for
k
=
1 to M
do
(8)   Estimates
b
k
t via detecting k
th LFB with classifier
in LK framework;
(9)   ComputeB
k
t via (15);
(10)  Retrain classifierf
i;
(11)end for
(12) EstimateB
t via (16);
(13) Adjust the set of offset via (14);
(14)for
k
=
1 to M
do
(15)  Check each LFB with corresponding classifier
(16)if
P
avg
k
<
λ
do
(17)   Updates
b
k
←
u
b
τ based on Section 4.4;
(18)   Generate positive samples(
X
k
+
,
Y
k
+
) and
negative  samples(
X
k
-
,
Y
k
-
) for the k
th LFB;
(19)    Train new classifierf
k for the k
th LFB
(20)break;
(21)end if
(22)end for
End


## 4.1. Set of Local Feature Blocks

Object is represented by LFBs, and thereby, object needs to be segmented into fragments. For simplicity, uniform segmentation is adopted in this paper as shown in Figure5.Figure 5
Uniform segmentation of object. Image (a) is object and image (b) is fragments.After segmentation, we select part blocks as LFBs. Assume object is divided intoK blocks; then we can obtain a candidate set of LFB set C
B
=
{
b
1
,
b
2
,
…
,
b
K
} with K candidate local feature blocks. For candidate LFB b
i, we compute its 2-bit Binary Patterns feature vector x
i. Then, scanning window method is used to compute the similar likelihood between feature vectors of input patch f
i and b
i, and the similarity is represented as L
i
j. M
L
i
=
max
⁡
(
L
i
j
) represents the highest similarity between b
i and all input patches. Finally, local feature block set S
B
=
{
s
b
1
,
s
b
2
,
…
,
s
b
M
} consists of M
(
M
<
K
) candidate local feature blocks, with smaller M
L
i.


## 4.2. Representations

In framet, B
t
=
(
C
t
,
R
t
,
W
t
,
H
t
) represents the object, where C
t and R
t are coordinates of center position and W
t and H
t are the sizes of object. s
b
k
t
=
(
c
k
t
,
r
k
t
,
w
k
t
,
h
k
t
) is the k
th LFB, where c
k
t and r
k
t are coordinates of center location and w
k
t and h
k
t are the sizes of local feature block. z
k
t
=
(
o
c
k
t
,
o
r
k
t
,
r
w
k
t
,
r
h
k
t
) represents the offset of the k
th LFB relative to object, where o
c
k
t and o
r
k
t are the offset of center coordinates and r
w
k
t and r
h
k
t are the rations of sizes between object and the k
th LFB. z
k
t can be determined by the following formulation:
(14)
o
c
k
t
=
C
t
-
c
k
t
,
o
r
k
t
=
R
t
-
r
k
t
,
r
w
k
t
=
W
t
w
k
t
,
r
h
k
t
=
H
t
h
k
t
.


## 4.3. Object Tracking

The tracked target is determined in initial frame (first frame) and segmented into fragments according to Section4.1. Then, we select local feature blocks and compute B
1, s
b
k
1, and z
k
1.Assume current frame ist. Each LFB is corresponded with a classifier via weighted P-N learning, and then we track each LFB. For the k
th LFB, s
b
k
t
+
1
=
(
c
k
t
+
1
,
r
k
t
+
1
,
w
k
t
+
1
,
h
k
t
+
1
) is used to represent its tracking result in frame t
+
1. By combining tracking results of each LFB and its offset z
k
t, we can obtain the corresponding object B
k
t
+
1
=
(
C
k
t
+
1
,
R
k
t
+
1
,
W
k
t
+
1
,
H
k
t
+
1
) via formulation the following formulation:
(15)
C
k
t
+
1
=
o
c
k
t
+
c
k
t
+
1
,
R
k
t
+
1
=
o
r
k
t
+
r
k
t
+
1
,
W
k
t
+
1
=
r
w
k
t
·
w
k
t
+
1
,
H
k
t
+
1
=
r
h
k
t
·
h
k
t
+
1
.
Each LFB determines a related candidate object. Object finally can be located by the following formulation:
(16)
B
t
+
1
=
1
M
·
∑
i
=
1
M
‍
B
i
t
+
1
,
where M is the number of LFBs. The entire process can be explained by Figure 6. An adjustment is needed for offset of each LFB relative to object. We divide new object region B
t
+
1 into fragments and compute the new offset of each LFB with prior LFBs based on formulation (14). After this, classifier of each LFB needs to be retrained via weighted P-N learning.Figure 6
The process of locating object. The green, pink, and blue patches are the local feature blockss
b
1
t
+
1, s
b
2
t
+
1, and s
b
3
t
+
1, and the green, blue, and pink dotted line rectangles are the corresponding tracking results B
1
t
+
1, B
2
t
+
1, and B
3
t
+
1. The red solid line rectangle represents the final tracking result B
t
+
1, which is located by tracking results of local feature blocks.Representation by LFBs has significant advantages. Firstly, compared with entire object, local feature block is more prone to recognition in background, and this guarantees the accuracy and stability of tracking. Besides, object is located by averaging all the tracking results of local feature blocks, which decreases tracking errors by counteracting positive and negative errors; therefore, the robustness of proposed algorithm is improved.


## 4.4. Updating Set of Local Feature Blocks

During the tracking process, update is essentially adaptive to complex environment variation. In this paper, the learning procedure is online, and then the main problem is how to handle the situation of local feature blocks being invalid. A strategy of replacing is adopted to solve this problem. During the tracking procedure, we make a real-time valid detection (in Section3.3) for all local feature blocks. If P
avg
≤
λ, local feature block is invalid. When certain local feature block is invalid, it will be replaced with an appropriate block, which is selected from the outside of the LFB set.LetU
B be the unused block set, and U
B
=
{
u
b
1
,
u
b
2
,
…
,
u
b
P
}
,
P
=
K
-
M
,
U
B
⋃
S
B
=
C
B. In current frame t, LFB s
b
k
t from S
B is invalid and needs to be replaced. We first segment object into blocks and obtain U
B. For block j from U
B, we compute similar likelihood l
j
t
=
sim
(
x
j
t
,
x
j
q
), where sim is function of computing similar likelihood, x
j
t is feature vector of block j in frame t, and x
j
q is feature vector of block j before it is used the last time in frame q (if block j is never used, q equals one). u
b
τ is used to replace s
b
k
t via the following formulation:
(17)
u
b
τ
=
arg
max
⁡
⁡
l
j
t
(
u
b
j
)
.
The whole update process can be illustrated as shown in Figure 7.The process of updating. The object is located by LFBs in (a) without occlusion. When occlusion occurs, yet each LFB is valid, the object can be tracked by LFBs exactly, as shown in (b). If occlusion happens and certain LFB is invalid, then it will be replaced and target still can be successfully located, as shown in (c), (d), (e), and (f).
(a)
(b)
(c)
(d)
(e)
(f)So far, we have introduced the overall procedure of the proposed tracking algorithm as shown in Algorithm1.Algorithm 1:Tracking based on proposed method.
Initialization:
(1) Segment object intoN blocks and obtain C
B
=
{
c
b
i
}
i
=
1
N;
(2) Select the set of LFBsS
B
=
{
s
b
k
}
k
=
1
M
(
M
<
N
);
(3) Compute the set of offset in first frame{
z
k
1
}
k
=
1
M;
(4) Generate positive samples(
X
k
+
,
Y
k
+
) and negative
samples(
X
k
-
,
Y
k
-
) for the k
t
h LFB;
(5) Train classifierf
k for the k
th LFB with data (
X
,
Y
) via
weighted P-N learning, whereX
=
{
X
k
+
,
X
k
-
},
andY
=
{
Y
k
+
,
Y
k
-
};
Object Tracking:
(6)for
t
=
2 to the end of the sequence
do
(7)for
k
=
1 to M
do
(8)   Estimates
b
k
t via detecting k
th LFB with classifier
in LK framework;
(9)   ComputeB
k
t via (15);
(10)  Retrain classifierf
i;
(11)end for
(12) EstimateB
t via (16);
(13) Adjust the set of offset via (14);
(14)for
k
=
1 to M
do
(15)  Check each LFB with corresponding classifier
(16)if
P
avg
k
<
λ
do
(17)   Updates
b
k
←
u
b
τ based on Section 4.4;
(18)   Generate positive samples(
X
k
+
,
Y
k
+
) and
negative  samples(
X
k
-
,
Y
k
-
) for the k
th LFB;
(19)    Train new classifierf
k for the k
th LFB
(20)break;
(21)end if
(22)end for
End


## 5. Experimental Results

In order to evaluate the performance of our tracking algorithm, we test our tracker on thirteen challenging image sequences. These sequences cover most challenging situations in visual tracking as shown in Table1. For comparison, we run six state-of-the-art tracking algorithms with the same initial position of object. These algorithms are l
1 tracking [2], FG tracking [8], IVT tracking [3], MIL tracking [5], TLD tracking [26], and CT tracking [6] approaches. Some representative results are shown in this section.Table 1
The tracking sequences used in our experiments.
Sequence
Frames
Main challenges
Cup
303
Direction variation and scale variation
DavidIndoor
770
Illumination, scale variation, and pose change
DavidOutdoor
569
Illumination and scale variation
Deer
71
Fast motion, motion blur, and background clutter
Juice
404
Direction variation and scale variation
Jumping
313
Fast motion and motion blur
Lemming
900
Fast motion, motion blur, and occlusion
Occlusion1
415
Occlusion
OneLSR
559
Scale variation and occlusion
OSOW2cor
320
Scale variation and pose change
PersonFloor
387
Scale variation and occlusion
Subway
175
Occlusion and background clutter
Woman
551
Occlusion, scale variation, and pose change


### 5.1. Quantitative Comparison

We evaluate the above-mentioned trackers via overlapping rate [29] as well as center location error, and the comparing results are shown in Tables 2 and 3.Table 2
Center location errors (in pixels). The best result is shown in bold and the second best in italic fonts.
l
1
FG
IVT
MIL
TLD
CT
Ours
Woman
136.69
106.21
110.46
114.02
148.66
107.74
3.77
DavidIndoor
57.52
—
15.78
26.43
16.19
31.83
10.34
Cup
4.09
7.02
2.23
4.78
3.86
5.62
2.47
Juice
4.39
—
1.80
11.07
4.57
47.97
3.07
Deer
132.59
97.28
206.68
231.29
45.83
240.36
7.02
Lemming
181.13
164.87
192.12
168.87
8.19
73.28
10.14
OneLSR
4.87
54.70
8.43
65.48
10.50
76.47
3.95
PersonFloor
37.63
68.15
27.06
37.13
62.94
26.89
9.52
Subway
153.60
8.06
124.60
137.86
5.26
11.50
3.19
OSOW2cor
3.06
5.22
1.62
13.83
7.50
8.08
3.97
DavidOutdoor
26.50
45.87
75.86
59.77
102.20
73.72
3.71
Occlusion1
8.85
8.60
21.81
31.91
35.43
37.96
7.11
Jumping
64.23
52.34
50.18
54.03
4.06
45.11
6.87
Average
62.70
56.21
64.51
73.57
35.01
60.50
5.78Table 3
Overlapping rate. Bold fonts indicate the best performance while the italic fonts indicate the second best ones.
l
1
FG
IVT
MIL
TLD
CT
Ours
Woman
0.06
0.19
0.14
0.15
0.04
0.15
0.72
DavidIndoor
0.21
—
0.3
0.45
0.52
0.39
0.47
Cup
0.64
0.67
0.61
0.77
0.63
0.73
0.62
Juice
0.73
—
0.79
0.64
0.66
0.07
0.69
Deer
0.04
0.07
0.03
0.04
0.34
0.04
0.65
Lemming
0.19
0.05
0.19
0.07
0.64
0.43
0.7
OneLSR
0.54
0.24
0.3
0.26
0.45
0.26
0.56
PersonFloor
0.47
0.25
0.53
0.47
0.21
0.6
0.64
Subway
0.15
0.61
0.17
0.09
0.68
0.5
0.67
OSOW2cor
0.59
0.43
0.63
0.39
0.5
0.4
0.62
DavidOutdoor
0.35
0.42
0.17
0.3
0.08
0.31
0.62
Occlusion1
0.71
0.72
0.5
0.55
0.48
0.41
0.66
Jumping
0.07
0.1
0.17
0.01
0.78
0.07
0.75
Average
0.37
0.36
0.35
0.32
0.46
0.33
0.64Figure8 shows the center location error of utilized tracker on thirteen test sequences. Overall, the tracker proposed in this paper outperforms the state-of-the-art algorithms.Quantitative evaluation of the trackers in terms of position errors (in pixels).
(a)
(b)
(c)


### 5.2. Qualitative Comparison

Heavy Occlusion. Occlusion is one of the most common yet crucial issues in visual tracking. We test four image sequences (Woman, Subway, PersonFloor, and Occlusion1) characterized in severe occlusion or long-time partial occlusion. Figure 9(a) demonstrates the robustness performance of proposed tracking method in handling occlusion. Object is represented by local feature blocks in proposed algorithm. When occlusion happens, other tracking algorithms cannot track object well because they are prone to update background into object. However, our tracker can employ a new unused block to replace the invalid local feature block when occlusion occurs, which can make local feature block set effective to continue tracking.Tracking results on various challenging sequences.
(a)
Woman, Subway, Occlusion1, andPersonFloor with heavy occlusion
(b)
OneLSR,OSOW2cor,Juice,andCupwith scale variation
(c)
Jumping,Lemming,andDeerwith fast motion and motion blur
(d)
DavidIndoorandDavidOutdoorwith illumination changesScale Variation. Figure 9(b) presents the tracking results on four image sequences (OneLSR, OSOW2cor, Juice, and Cup) with large scale variation, even more with slight rotation. Our tracker can tail object throughout the whole sequences, which can be attributed to the discriminative classifier based on weighted P-N learning. We also observe that local feature blocks can better represent object, which makes the tracker focus on the stable part of the object.Fast Motion and Motion Blur. Figure 9(c) demonstrates experimental results on three challenging sequences (Deer, Lemming, and Jumping). Because the target undergoes fast and abrupt motion, it is more prone to cause blur, which causes drifting problem. It is worth noticing that the suggested approach in this paper performs better than other algorithms. When motion blur occurs, our tracker can guarantee that the object’s local feautres are still available. The advantages of using local feature blocks to represent object are shown incisively and vividly. By combining improved P-N learning and local feature blocks, we can obtain a discriminative classifier of stable object parts, which can locate the object. Then we track each local feature block, respectively, and determine the object based on tracking results of local feature blocks.Illumination Variation. Illumination is a critical factor in visual tracking. Two typical image sequences (DavidIndoor and DavidOutdoor) are employed to test our tracker as shown in Figure 9(d). When illumination varies, some local regions of target are insensitive actually. Our tracker captures these insensitive regions to track local areas of object and further locate entire object via local tracking information.


## 5.1. Quantitative Comparison

We evaluate the above-mentioned trackers via overlapping rate [29] as well as center location error, and the comparing results are shown in Tables 2 and 3.Table 2
Center location errors (in pixels). The best result is shown in bold and the second best in italic fonts.
l
1
FG
IVT
MIL
TLD
CT
Ours
Woman
136.69
106.21
110.46
114.02
148.66
107.74
3.77
DavidIndoor
57.52
—
15.78
26.43
16.19
31.83
10.34
Cup
4.09
7.02
2.23
4.78
3.86
5.62
2.47
Juice
4.39
—
1.80
11.07
4.57
47.97
3.07
Deer
132.59
97.28
206.68
231.29
45.83
240.36
7.02
Lemming
181.13
164.87
192.12
168.87
8.19
73.28
10.14
OneLSR
4.87
54.70
8.43
65.48
10.50
76.47
3.95
PersonFloor
37.63
68.15
27.06
37.13
62.94
26.89
9.52
Subway
153.60
8.06
124.60
137.86
5.26
11.50
3.19
OSOW2cor
3.06
5.22
1.62
13.83
7.50
8.08
3.97
DavidOutdoor
26.50
45.87
75.86
59.77
102.20
73.72
3.71
Occlusion1
8.85
8.60
21.81
31.91
35.43
37.96
7.11
Jumping
64.23
52.34
50.18
54.03
4.06
45.11
6.87
Average
62.70
56.21
64.51
73.57
35.01
60.50
5.78Table 3
Overlapping rate. Bold fonts indicate the best performance while the italic fonts indicate the second best ones.
l
1
FG
IVT
MIL
TLD
CT
Ours
Woman
0.06
0.19
0.14
0.15
0.04
0.15
0.72
DavidIndoor
0.21
—
0.3
0.45
0.52
0.39
0.47
Cup
0.64
0.67
0.61
0.77
0.63
0.73
0.62
Juice
0.73
—
0.79
0.64
0.66
0.07
0.69
Deer
0.04
0.07
0.03
0.04
0.34
0.04
0.65
Lemming
0.19
0.05
0.19
0.07
0.64
0.43
0.7
OneLSR
0.54
0.24
0.3
0.26
0.45
0.26
0.56
PersonFloor
0.47
0.25
0.53
0.47
0.21
0.6
0.64
Subway
0.15
0.61
0.17
0.09
0.68
0.5
0.67
OSOW2cor
0.59
0.43
0.63
0.39
0.5
0.4
0.62
DavidOutdoor
0.35
0.42
0.17
0.3
0.08
0.31
0.62
Occlusion1
0.71
0.72
0.5
0.55
0.48
0.41
0.66
Jumping
0.07
0.1
0.17
0.01
0.78
0.07
0.75
Average
0.37
0.36
0.35
0.32
0.46
0.33
0.64Figure8 shows the center location error of utilized tracker on thirteen test sequences. Overall, the tracker proposed in this paper outperforms the state-of-the-art algorithms.Quantitative evaluation of the trackers in terms of position errors (in pixels).
(a)
(b)
(c)


## 5.2. Qualitative Comparison

Heavy Occlusion. Occlusion is one of the most common yet crucial issues in visual tracking. We test four image sequences (Woman, Subway, PersonFloor, and Occlusion1) characterized in severe occlusion or long-time partial occlusion. Figure 9(a) demonstrates the robustness performance of proposed tracking method in handling occlusion. Object is represented by local feature blocks in proposed algorithm. When occlusion happens, other tracking algorithms cannot track object well because they are prone to update background into object. However, our tracker can employ a new unused block to replace the invalid local feature block when occlusion occurs, which can make local feature block set effective to continue tracking.Tracking results on various challenging sequences.
(a)
Woman, Subway, Occlusion1, andPersonFloor with heavy occlusion
(b)
OneLSR,OSOW2cor,Juice,andCupwith scale variation
(c)
Jumping,Lemming,andDeerwith fast motion and motion blur
(d)
DavidIndoorandDavidOutdoorwith illumination changesScale Variation. Figure 9(b) presents the tracking results on four image sequences (OneLSR, OSOW2cor, Juice, and Cup) with large scale variation, even more with slight rotation. Our tracker can tail object throughout the whole sequences, which can be attributed to the discriminative classifier based on weighted P-N learning. We also observe that local feature blocks can better represent object, which makes the tracker focus on the stable part of the object.Fast Motion and Motion Blur. Figure 9(c) demonstrates experimental results on three challenging sequences (Deer, Lemming, and Jumping). Because the target undergoes fast and abrupt motion, it is more prone to cause blur, which causes drifting problem. It is worth noticing that the suggested approach in this paper performs better than other algorithms. When motion blur occurs, our tracker can guarantee that the object’s local feautres are still available. The advantages of using local feature blocks to represent object are shown incisively and vividly. By combining improved P-N learning and local feature blocks, we can obtain a discriminative classifier of stable object parts, which can locate the object. Then we track each local feature block, respectively, and determine the object based on tracking results of local feature blocks.Illumination Variation. Illumination is a critical factor in visual tracking. Two typical image sequences (DavidIndoor and DavidOutdoor) are employed to test our tracker as shown in Figure 9(d). When illumination varies, some local regions of target are insensitive actually. Our tracker captures these insensitive regions to track local areas of object and further locate entire object via local tracking information.


## 6. Conclusions

In this paper, we propose a part-based visual tracking algorithm with online weighted P-N learning. An online P-N learning is presented by assigning weight (property weight and classification weight) to each sample in training sample set, which can decrease classification errors and can improve the discriminative power of classifier. Firstly, the target is segmented into fragments, and parts of them are chosen to be local feature blocks to represent object. We then train classifier for each LFB with weighted P-N learning, obtain the corresponding classifier, respectively, and track each LFB independently within the framework of LK optical flow. In addition, a substitute strategy is adopted to update dynamically the set of LFBs, which ensures robust tracking. Experimental results demonstrate that our algorithm outperforms state-of-the-art trackers. However, our algorithm fails to track object exactly in some scenes. If the tracked target is nonrigid and has an extremely heavy deformation or is fully occluded for long time, the performance of proposed tracker drops.



---

*Source: 402185-2014-07-15.xml*