# Visual Tracking Based on Discriminative Compressed Features

Authors: Wei Liu; Hui Wang
Journal: Advances in Multimedia
 (2018)

---

## Abstract

Visual tracking is a challenging research topic in the field of computer vision with many potential applications. A large number of tracking methods have been proposed and achieved designed tracking performance. However, the current state-of-the-art tracking methods still can not meet the requirements of real-world applications. One of the main challenges is to design a good appearance model to describe the target’s appearance. In this paper, we propose a novel visual tracking method, which uses compressed features to model target’s appearances and then uses SVM to distinguish the target from its background. The compressed features were obtained by the zero-tree coding on multiscale wavelet coefficients extracted from an image, which have both the low dimensionality and discriminate ability and therefore ensure to achieve better tracking results. The experimental comparisons with several state-of-the-art methods demonstrate the superiority of the proposed method.


---

## Body

## 1. Introduction

Visual tracking aims at locating the target of interest from an image sequence, which is one of the most activated research topics in the field of computer vision with many potential applications such as video surveillance, human-computer interaction, navigation, and automatic driving. It has attracted increasing interest in the past few decades [1–16]. However, due to a variety of challenging factors such as illumination changes, pose deformation, and occlusion, the performance of visual tracking is still far away from requirements in practical applications. The main difficulty is that it is not easy to design a good appearance modeling method, which is not only good at distinguishing the target from its background but also being robust to the above-mentioned appearance changes. Finding a good appearance modeling is a challenging problem in many visual applications such as image classification [17–19] and video recognition [20–22].In the literature, there are a variety of visual tracking methods with focus on developing effective appearance modeling methods. Most of these methods can be classified into two groups: generative methods and discriminative methods. The former learns generative features from samples that only contain the target, whose purpose is to represent the target as accurate as possible. The latter learns discriminative features from samples including both the target and its background, which usually involves solving an optimization function. To achieve better tracking performance, discriminative methods attracted more attention.In this paper, to overcome the challenges caused by low contrast, illuminative changes, and scale changes, we propose a novel tracking method using discriminative compressed features, which is real-time and able to process multiple scales of the target. The main idea of the proposed method is that it combines compressive sensing and multiscale texture transformation to extract compressed texture features and then uses SVM to classify the target from its background. The compressed features have both the low dimensionality and discriminate ability and therefore ensure to achieve better tracking results. The experimental comparisons with several state-of-the-art methods demonstrate the superiority of the proposed method.The rest of this paper is organized as follows. In Section2, we review the work closely related to our proposed approach. Section 3 gives a detailed description of the proposed tracking method. Experimental results are reported and analyzed in Section 6. We conclude this paper in Section 6.


## 2. Related Work

In the past decades, there are many tracking methods that have been proposed, which can be roughly divided into generative methods and discriminative methods. The former focuses on modeling the appearance of the tracked target and then finds the candidate that is the most similar to the target template as the tracking result. The representative methods include those trackers based on sparse representation [23–29]. In [29], sparse coding is used to extract features from sampled patches. The local sparse features are then pooled into a global representation. In [28], an online learning sparse representation is proposed for visual tracking to handle occlusion. In [25], a joint sparse representation framework is used to combine multi-cue features for visual tracking. Since features from different cues describe the tracked target from different aspects, more robust tracking results can be obtained when multi-cue features are used. In [23], a biologically inspired appearance model is proposed to model target appearance, which is also based on features extracted using sparse coding.The discriminative methods learn a binary classifier, which is then used to classify a candidate as the target or background [5, 8, 14, 16, 30–34]. In [30], Yakut and Kehtarnavaz proposed to track ice-hockey pucks by combining three pieces of information in ice-hockey video frames using an adaptive gray-level thresholding method. In [31], Topkaya et al. proposed a multiple object tracking method using tracklet clustering, which first obtains short yet reliable tracklets and then clusters the tracklets over time based on color and spatial and temporal attributes. In [32], Wang and Zhao proposed an adaptive appearance model called Principal Component-Canonical Correlation Analysis (P3CA) to extract discriminative features for object tracking. In [14], Qi et al. propose a CNN based tracking method, which uses correlation filters to construct six weak trackers on outputs of six CNN layers. These weak trackers are then adaptively combined by a Normal Hedge algorithm. In [34], a further improved method is proposed which uses a SNT to compute the loss of each weak tracker, which achieves better tracking performance.


## 3. Discriminative Compressed Features

### 3.1. Multiscale Wavelet Transformation

Multiscale wavelet is a kind of wavelet which consists of more than two scale functions. It preserves the local properties of time-frequency domains while overcoming the drawbacks of a single wavelet and therefore has more properties of different frequencies. In this paper, we choose the GHM multiscale wavelet [35], which can be obtained by recursively calculating as follows:(1)vj,k=∑mGm-2kvj-1,m(2)wj,k=∑mHm-2kvj-1,mwhere vj,k and wj,k are low-frequency coefficients and high-frequency coefficients of the jth scale of the input signal, respectively. vj-1,m denotes the low-frequency coefficients of the (j-1)th scale; k and m are the indices of the current scales, which are dependent on the input image. The multiwavelet filters are defined as(3)G0=35245-120-3102G1=352092012(4)G2=00920-3102G3=00-1200(5)H0=110-12-32123H1=11092-102-920(6)H2=11092-3292-3H3=110-120-120


### 3.2. Compressed Multiscale Features

It is easy to obtain low-frequency components and high-frequency components after the signals are filtered by wavelet transformation. In general, most energy of the signal is in the low-frequency components. In contrast, high-frequency components of the signal reflect the details of the input image. Therefore, the simplest way of compressing the input image is to set the high-frequency coefficients to be zero when reconstructing the input image using wavelet transformation. The other option is to set the high-frequency coefficients of some local regions to be zero or to set the high-frequency coefficients based on a threshold, which will cause severe loss of image details, blurred images after compression, or loss of image information.Wavelet transformation is able to composite the input image at different scales. More importantly, the subimage at each resolution has different frequency properties and different orientation selections. Therefore, it can be used to encode different information of the input image at different scales.It is widely thought of the fact that the targets in a video sequence are redundant in both spatial and frequency domains. The former indicates the adjacent pixels have spatial correlation. The latter indicates that the adjacent frequencies of a pixel have some kinds of correlation. On the other hand, the statistical features of image signals indicate that large coefficients always exist in low-frequency regions and therefore small bits can be assigned to those small coefficients or they will not be transmitted at all. It will cause high compression rates and very small information loss.The compression method based on multiscale wavelet transformation applies the zero-tree coding to compression of high spectral images. The principle behind this method is that it exploits the structure correlation of high spectral images to construct only one effective (shared) image and then further determine the positions of nonzeros of multiscale wavelet coefficients. The shared image is obtained by combining multiscale frequency coefficients and therefore removes spatial redundancy and frequency redundancy with the purpose of improving compression efficiency.The one-dimensional wavelet transformation filters the input signal by low-pass filtering and high-pass filtering and then obtains low-frequency components and high-frequency components by downsampling. According to Mallat algorithm, two-dimensional wavelet transformation can be implemented by several one-dimensional wavelet transformation and obtain low-frequency and high-frequency components, respectively. Given an input image with m rows and n columns, the process of 2D wavelet transformation is that it first decomposes the input image along its each row using 1D wavelet transformation, which will obtain L and H two parts. The second step is to decompose the L and H parts along its column using 1D wavelet transformation. With these two steps, the input image will get four parts (LL, HL, LH, and HH). The second level, third level, or higher level’s wavelet transformation can be obtained by using such a process on the former level. Therefore, the wavelet transformation is an iterative process.To meet the real-time requirements, the dimensionality of appearance features should not be too high. To meet this requirement, in this paper, we adopt compressive sensing to reduce the dimensionality of high-dimensional appearance features. Letu∈RD be the wavelet features and Γ be a random matrix computed using the same method as in [26]. The compressed features v∈Rd can be computed as v=Γu.


## 3.1. Multiscale Wavelet Transformation

Multiscale wavelet is a kind of wavelet which consists of more than two scale functions. It preserves the local properties of time-frequency domains while overcoming the drawbacks of a single wavelet and therefore has more properties of different frequencies. In this paper, we choose the GHM multiscale wavelet [35], which can be obtained by recursively calculating as follows:(1)vj,k=∑mGm-2kvj-1,m(2)wj,k=∑mHm-2kvj-1,mwhere vj,k and wj,k are low-frequency coefficients and high-frequency coefficients of the jth scale of the input signal, respectively. vj-1,m denotes the low-frequency coefficients of the (j-1)th scale; k and m are the indices of the current scales, which are dependent on the input image. The multiwavelet filters are defined as(3)G0=35245-120-3102G1=352092012(4)G2=00920-3102G3=00-1200(5)H0=110-12-32123H1=11092-102-920(6)H2=11092-3292-3H3=110-120-120


## 3.2. Compressed Multiscale Features

It is easy to obtain low-frequency components and high-frequency components after the signals are filtered by wavelet transformation. In general, most energy of the signal is in the low-frequency components. In contrast, high-frequency components of the signal reflect the details of the input image. Therefore, the simplest way of compressing the input image is to set the high-frequency coefficients to be zero when reconstructing the input image using wavelet transformation. The other option is to set the high-frequency coefficients of some local regions to be zero or to set the high-frequency coefficients based on a threshold, which will cause severe loss of image details, blurred images after compression, or loss of image information.Wavelet transformation is able to composite the input image at different scales. More importantly, the subimage at each resolution has different frequency properties and different orientation selections. Therefore, it can be used to encode different information of the input image at different scales.It is widely thought of the fact that the targets in a video sequence are redundant in both spatial and frequency domains. The former indicates the adjacent pixels have spatial correlation. The latter indicates that the adjacent frequencies of a pixel have some kinds of correlation. On the other hand, the statistical features of image signals indicate that large coefficients always exist in low-frequency regions and therefore small bits can be assigned to those small coefficients or they will not be transmitted at all. It will cause high compression rates and very small information loss.The compression method based on multiscale wavelet transformation applies the zero-tree coding to compression of high spectral images. The principle behind this method is that it exploits the structure correlation of high spectral images to construct only one effective (shared) image and then further determine the positions of nonzeros of multiscale wavelet coefficients. The shared image is obtained by combining multiscale frequency coefficients and therefore removes spatial redundancy and frequency redundancy with the purpose of improving compression efficiency.The one-dimensional wavelet transformation filters the input signal by low-pass filtering and high-pass filtering and then obtains low-frequency components and high-frequency components by downsampling. According to Mallat algorithm, two-dimensional wavelet transformation can be implemented by several one-dimensional wavelet transformation and obtain low-frequency and high-frequency components, respectively. Given an input image with m rows and n columns, the process of 2D wavelet transformation is that it first decomposes the input image along its each row using 1D wavelet transformation, which will obtain L and H two parts. The second step is to decompose the L and H parts along its column using 1D wavelet transformation. With these two steps, the input image will get four parts (LL, HL, LH, and HH). The second level, third level, or higher level’s wavelet transformation can be obtained by using such a process on the former level. Therefore, the wavelet transformation is an iterative process.To meet the real-time requirements, the dimensionality of appearance features should not be too high. To meet this requirement, in this paper, we adopt compressive sensing to reduce the dimensionality of high-dimensional appearance features. Letu∈RD be the wavelet features and Γ be a random matrix computed using the same method as in [26]. The compressed features v∈Rd can be computed as v=Γu.


## 4. Discriminative SVM Tracking

SVM is for classic binary pattern classification since it was proposed by Vapnik in 1995. In this paper, we use SVM as our tracking model.


### 4.1. SVM Tracking

To classify the target from its background, our tracking method tries to find a hyperplane in the D-dimensional compressed feature space to distinguish the features of the target and its background.To achieve this aim, the optimization objective is to maximize the classifier’s margin in the feature space. In other words, we need to meet the following conditions:(7)xi·w+b≥0ifyi=+1(8)xi·w+b≤0ifyi=-1 where yi is the class label of the ith sample. For example, if the sample is target, yi=+1. Otherwise, if the sample is background, yi=-1.Given training samples and their corresponding labels, we first extract compressed features from each sample using the method introduced in Section3. The features with their labels can then be fed to SVM to train SVM’s parameters. In the tracking stage, for each target candidate, we can also extract the compressed features using the same method as like in the training stage. Then we can feed the extracted features to SVM to predicate its label. If the features are classified as +1, it is considered as the potential target. Otherwise, it is not considered as the potential target. The final target is selected as the potential target candidate with the largest probability.


### 4.2. Model Update

To make the proposed tracker adapt to target appearance changes over time, the tracker needs to be updated online. To this aim, we update the model using the collected positive and negative samples. In particular, we collect a set of positive and negative samples at timet. Using the proposed appearance model, we can extract the compressed features for all positive and negative samples. Then the SVM model can be updated as(9)ul1=λul1+1-λul1(10)ul0=λul0+1-λul0(11)δl1=λδl12+1-λδl12+λ1-λul1-u12(12)δl0=λδl02+1-λδl02+λ1-λul0-u02where λ denotes the learning rate, which controls the speed of model updating.(13)u1=1α∑k=1αv1,lk(14)u0=1α∑k=1αv0,lk(15)δ1=1α∑k=1αv1,lk-u12(16)δ0=1α∑k=1αv0,lk-u02


## 4.1. SVM Tracking

To classify the target from its background, our tracking method tries to find a hyperplane in the D-dimensional compressed feature space to distinguish the features of the target and its background.To achieve this aim, the optimization objective is to maximize the classifier’s margin in the feature space. In other words, we need to meet the following conditions:(7)xi·w+b≥0ifyi=+1(8)xi·w+b≤0ifyi=-1 where yi is the class label of the ith sample. For example, if the sample is target, yi=+1. Otherwise, if the sample is background, yi=-1.Given training samples and their corresponding labels, we first extract compressed features from each sample using the method introduced in Section3. The features with their labels can then be fed to SVM to train SVM’s parameters. In the tracking stage, for each target candidate, we can also extract the compressed features using the same method as like in the training stage. Then we can feed the extracted features to SVM to predicate its label. If the features are classified as +1, it is considered as the potential target. Otherwise, it is not considered as the potential target. The final target is selected as the potential target candidate with the largest probability.


## 4.2. Model Update

To make the proposed tracker adapt to target appearance changes over time, the tracker needs to be updated online. To this aim, we update the model using the collected positive and negative samples. In particular, we collect a set of positive and negative samples at timet. Using the proposed appearance model, we can extract the compressed features for all positive and negative samples. Then the SVM model can be updated as(9)ul1=λul1+1-λul1(10)ul0=λul0+1-λul0(11)δl1=λδl12+1-λδl12+λ1-λul1-u12(12)δl0=λδl02+1-λδl02+λ1-λul0-u02where λ denotes the learning rate, which controls the speed of model updating.(13)u1=1α∑k=1αv1,lk(14)u0=1α∑k=1αv0,lk(15)δ1=1α∑k=1αv1,lk-u12(16)δ0=1α∑k=1αv0,lk-u02


## 5. Experiment Results

The target tracking is implemented in a particle filter framework. Several sequences from the OTB100 dataset have been chosen to evaluate the proposed tracking method. At the first frame, the target is initialized manually. Of course, the target can be initialized by a detector when the method is applied in real systems. After the target is initialized, a set of particles are sampled around the target. Whether each particle is considered as the target or not is based on the output of SVM scoring. In the next frame, the particles are sampled using the tracking result in the last frame as mean and a predefined covariance. The process is repeated frame by frame. The flowchart of the proposed tracking method is shown in Figure1.Figure 1
The flowchart of the proposed tracking method.To test the performance of the proposed method, we compared the proposed method to several state-of-the-art trackers including TLD [36], CXT [1], Struck [37], L1APG [38], and MTT [39]. By quantitatively and qualitatively analyzing the experimental results, we demonstrate the outstanding performance of the proposed method.Two frame based metrics widely used in tracking performance evaluation are(1) center location error, which is defined as the Euclidean distance between the central location of the tracked target and the manually labeled ground-truthed position; (2) bounding box overlap which is the ratio of the areas of the intersection and the union of the bounding box indicating the tracked subject and the ground-truthed bounding box. To measure the overall performance of a tracker on a test sequence, success rate and precision score are adopted. The former is computed as the percentage of image frames, which have a bounding box overlap larger than a given threshold. The latter is the percentage of image frames, which have a central position error less than a given threshold. In each case, when multiple thresholds are used, a curve is drawn to show how success rates or precision scores are affected by different thresholds. These curves are, namely, success plot and precious plot, respectively. In practical evaluations, we average the curves of a tracker over all the sequences, which have the same challenge and show a curve for each challenge item rather than a test sequence. In addition, we use the area under curve (AUC) of the success plot to quantitatively measure the overall performance of a tracker on a challenge item.


### 5.1. Quantitative Comparison

The overall precision plots and success plots are shown in Figure2, from which we can see that the proposed method outperforms other methods in terms of the overall precision plots and success plots.Figure 2
Overall precision plots and success plots on the test sequences.


### 5.2. Qualitative Comparison

To further show the superiority of the proposed method, we show several examples of tracking results on Figures3 and 4. As we can see from Figure 3, the proposed tracker outperforms other trackers on several representative frames on two sequences. More tracking results are shown in Figure 4, from which we can see that the proposed tracker also achieves the best tracking performance.Figure 3
Examples of tracking results on representative frames of two sequences.Figure 4
Examples of tracking results on representative frames of other four sequences.


## 5.1. Quantitative Comparison

The overall precision plots and success plots are shown in Figure2, from which we can see that the proposed method outperforms other methods in terms of the overall precision plots and success plots.Figure 2
Overall precision plots and success plots on the test sequences.


## 5.2. Qualitative Comparison

To further show the superiority of the proposed method, we show several examples of tracking results on Figures3 and 4. As we can see from Figure 3, the proposed tracker outperforms other trackers on several representative frames on two sequences. More tracking results are shown in Figure 4, from which we can see that the proposed tracker also achieves the best tracking performance.Figure 3
Examples of tracking results on representative frames of two sequences.Figure 4
Examples of tracking results on representative frames of other four sequences.


## 6. Conclusion

In this paper, we propose to use compressed features to model the tracked target’s appearance and then use SVM to perform tracking. The experimental results indicate the proposed method outperforms several state-of-the-art methods. The advantages of the proposed method are twofold:(1) It is good at handling scale changes of the target over time because the used features are obtained by multiscale wavelet transformation. (2) The speed of the proposed method can achieve real-time because the dimensionality of the used features was reduced by compressed sensing techniques.



---

*Source: 7481645-2018-08-01.xml*