# Scale-Adaptive Context-Aware Correlation Filter with Output Constraints for Visual Target Tracking

Authors: Jingxiang Xu; Xuedong Wu; Zhiyu Zhu; Kaiyun Yang; Yanchao Chang; Zhaoping Du; Zhengang Wan; Lili Gu
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

Context-aware correlation filter tracker is one of the most advanced target trackers, and it has significant improvement in tracking accuracy and success rate compared with traditional trackers. However, because the complexity of background in the process of tracking can lead to inaccurate output response of target tracking, an accurate tracking model is difficult to be established. Moreover, the drift problem is easy to occur during the tracking process due to the imprecise tracking model, especially when the target has large area occlusion, fast motion, and deformation. Aiming at the drift problem in the target tracking process, a novel algorithm is proposed in this paper. The developed method derives the specific representation of constraint output by assuming that the output response is Gaussian distribution, and a variable update parameter is obtained based on the output constraint relationship at first, then the tracking filter is selectively updated with changeable update parameters and fixed update parameters, and finally, the target scale is updated with maximizing posterior probability distribution. The effectiveness of developed algorithm is verified by comparing with other trackers on OTB-50 and OTB-100 evaluation benchmark datasets, and the experimental results have shown that the suggested tracker has higher overall object tracking performance than other trackers.


---

## Body

## 1. Introduction

With the rapid development of computer technology, computer vision plays an increasingly important role in our lives [1, 2]. The scope of computer vision research is quite extensive, including face recognition [3], vehicle or pedestrian detection [4, 5], target tracking [6, 7], and image generation [8–11]. Visual target tracking has become one of the most influential research fields in computer vision because it is widely used in video surveillance [12], intelligent transportation, and military guidance [13, 14]. With the successful application of visual tracking technology in human life, visual object tracking is used to more and more complex environment, such as illumination variation, occlusions, fast motion, and deformation and background clutter, and these complex factors bring great challenges to the stable tracking of targets [15–18].The target tracking algorithm can be divided into the production tracking algorithm and discriminant tracking algorithm according to whether the background information of the target is utilized or not [19–21]. The production tracking algorithm uses only target information and discards background information. Although the production algorithm can shorten the computation time, it gives up useful background information which can lead to lower accuracy of tracking the target [22]. The discriminating tracking algorithm takes the tracking problem as a binary classification problem, and the target area is marked with a positive sample and the background region is signed with a negative sample in the current frame [23]. The target area and the background area are distinguished by a classifier, which is obtained by training a large number of samples using a classification method of ridge regression [24].Because the discriminant tracking algorithm has better tracking performance than the production tracking algorithm, it has been extensively used for target tracking in recent years. The context-aware correlation filtering (CACF) tracker is a discriminant tracking algorithm, and it has better tracking performance than many other algorithms for dealing with some complex targets [25]. The algorithm uses target information and background information around target area as sample input and takes target position as sample output in the current frame. The target position is determined by using the correlation filter and the target position information of the previous frame. This algorithm can improve tracking accuracy and ensure real-time target tracking. However, the target is prone to drift and causes tracking to fail when the CACF tracker tracks a fast-moving target. Moreover, because the CACF tracker uses a fixed learning rate to update the model, it is easy to cause the model to be updated inaccurately when the tracked target has occlusion or deformation. This inaccurate model update method may cause the target to drift in the tracking process.Although there are many trackers that have achieved excellent results in dealing with some of these problems, there are still some problems to be solved [26]. Aiming at the drift problem that is easy to occur in the tracking process, this paper proposes the scale adaptive context-aware filter with constraint for output response (OURS) to reduce target drift. The presented method in this paper mainly includes three innovations: (1) a variable parameter is found based on Gaussian distribution and correlation filtering at first, and then the filter is updated using the obtained variable parameter by assuming that the output response is Gaussian distribution. (2) The variable updating parameter with output response under Gaussian constraint and the fixed updating parameter are used to selectively update the filter. (3) The maximum posterior probability distribution is used to update the adaptive scale of the target.


## 2. Related Work

### 2.1. Correlation Filter Tracker

The correlation filtering algorithm can greatly reduce computational complexity by converting the time domain calculation into the frequency domain calculation, and it attracts attention of many researchers and applies it to visual target tracking. The principle of the correlation filtering algorithm is to produce a correlation response peak when an interesting object is encountered. The correlation filter has great application in the target tracking field because the tracking speed of traditional algorithms is slow. Bolme et al. [27] applied the correlation filter to target tracking for the first time and proposed the minimum output sum of the squared error tracker (MOSSE), and the tracker is obtained by calculating the least squared error between the actual output and the expected output. Then, many researchers were inspired by the MOSSE tracker and proposed a variety of improved target tracking algorithms. Henriques et al. [28] suggested the loop structure kernel correlation tracker (CSK) to improve the target tracking speed by using intensive sampling to get samples. Yuan et al. [29] proposed a metric learning model for visual tracking in the relevant filtering framework, which used the metric learning function to solve the target size problem. Ou et al. [30] developed a new method for selecting representative samples based on the coefficient-constrained model, which regarded the template as a linear combination of representative samples.In order to further improve the target tracking accuracy, Danelljan et al. [31] presented a real-time visual tracker with adaptive color attributes (CN) by embedding multichannel color attributes into kernel space to obtain an adaptive correlation filtering method. In order to deal with the impact of illumination variation on target tracking, Henriques et al. [32] developed the kernel correlation filter with high-speed tracking (KCF) by applying the histogram of oriented gradient (HOG) feature to the correlation filtering algorithm. The KCF tracker performs multichannel expansion of the linear correlation filter by introducing a linear kernel function, so the computational efficiency and the tracking performance were greatly improved.The traditional correlation filtering algorithm uses a fixed-size window to construct tracker that can estimate the displacement change of tracking target, but it cannot effectively handle the scale change of the target. In response to changes of the target scale, Li et al. [33] proposed a scale-adaptive kernel correlation filtering tracker (SAMF) by fusing color features and HOG features as the new feature of the tracker and determined the optimal target position by comparing the maximum output amplitude with different scales. Taking the advantages of color features and HOG features, Bertinetto et al. [34] suggested a real-time tracking color supplement learning tracker that uses features consisted of color features and HOG features.The successful application of correlation filter in target tracking can greatly improves the computational efficiency of the tracker, but there are still some unresolved problems, such as the drift problem is caused by fast motion, occlusion, and deformation.


### 2.2. Context-Aware Trackers

Context information can provide very significant ancillary information for tracking the target. The context-aware tracker not only selects the target area as sample but also selects background information around the target area as context area [35]. The whole sample space of the context-aware algorithm includes target area sample and context area sample. The specific situation is shown in Figure 1.Figure 1
Illumination of context-aware area.The context information may provide important supplementary information to target detection and tracking, and it can better identify the target’s state. A context-aware visual tracking tracker (CAT) that found auxiliary target by online learning and provided context information for the target was proposed [36]. The CAT tracker can reduce uncertainty in the target tracking process and improve tracking performance. In order to effectively suppress the interference influence, a context-aware sparse tracker (CEST) that used context particle information to create dynamically updating dictionary template was suggested [37].Although the CAT has achieved some successful results in target tracking, it is still unable to deal with the drift problem caused by fast motion, occlusion, and deformation. In order to solve the drift problem of tracking process, this paper proposes a scale adaptive context-aware correlation filter with constraints for output response. The experimental results have shown that the proposed algorithm can reduce the drift of target tracking and improve the target tracking performance.


## 2.1. Correlation Filter Tracker

The correlation filtering algorithm can greatly reduce computational complexity by converting the time domain calculation into the frequency domain calculation, and it attracts attention of many researchers and applies it to visual target tracking. The principle of the correlation filtering algorithm is to produce a correlation response peak when an interesting object is encountered. The correlation filter has great application in the target tracking field because the tracking speed of traditional algorithms is slow. Bolme et al. [27] applied the correlation filter to target tracking for the first time and proposed the minimum output sum of the squared error tracker (MOSSE), and the tracker is obtained by calculating the least squared error between the actual output and the expected output. Then, many researchers were inspired by the MOSSE tracker and proposed a variety of improved target tracking algorithms. Henriques et al. [28] suggested the loop structure kernel correlation tracker (CSK) to improve the target tracking speed by using intensive sampling to get samples. Yuan et al. [29] proposed a metric learning model for visual tracking in the relevant filtering framework, which used the metric learning function to solve the target size problem. Ou et al. [30] developed a new method for selecting representative samples based on the coefficient-constrained model, which regarded the template as a linear combination of representative samples.In order to further improve the target tracking accuracy, Danelljan et al. [31] presented a real-time visual tracker with adaptive color attributes (CN) by embedding multichannel color attributes into kernel space to obtain an adaptive correlation filtering method. In order to deal with the impact of illumination variation on target tracking, Henriques et al. [32] developed the kernel correlation filter with high-speed tracking (KCF) by applying the histogram of oriented gradient (HOG) feature to the correlation filtering algorithm. The KCF tracker performs multichannel expansion of the linear correlation filter by introducing a linear kernel function, so the computational efficiency and the tracking performance were greatly improved.The traditional correlation filtering algorithm uses a fixed-size window to construct tracker that can estimate the displacement change of tracking target, but it cannot effectively handle the scale change of the target. In response to changes of the target scale, Li et al. [33] proposed a scale-adaptive kernel correlation filtering tracker (SAMF) by fusing color features and HOG features as the new feature of the tracker and determined the optimal target position by comparing the maximum output amplitude with different scales. Taking the advantages of color features and HOG features, Bertinetto et al. [34] suggested a real-time tracking color supplement learning tracker that uses features consisted of color features and HOG features.The successful application of correlation filter in target tracking can greatly improves the computational efficiency of the tracker, but there are still some unresolved problems, such as the drift problem is caused by fast motion, occlusion, and deformation.


## 2.2. Context-Aware Trackers

Context information can provide very significant ancillary information for tracking the target. The context-aware tracker not only selects the target area as sample but also selects background information around the target area as context area [35]. The whole sample space of the context-aware algorithm includes target area sample and context area sample. The specific situation is shown in Figure 1.Figure 1
Illumination of context-aware area.The context information may provide important supplementary information to target detection and tracking, and it can better identify the target’s state. A context-aware visual tracking tracker (CAT) that found auxiliary target by online learning and provided context information for the target was proposed [36]. The CAT tracker can reduce uncertainty in the target tracking process and improve tracking performance. In order to effectively suppress the interference influence, a context-aware sparse tracker (CEST) that used context particle information to create dynamically updating dictionary template was suggested [37].Although the CAT has achieved some successful results in target tracking, it is still unable to deal with the drift problem caused by fast motion, occlusion, and deformation. In order to solve the drift problem of tracking process, this paper proposes a scale adaptive context-aware correlation filter with constraints for output response. The experimental results have shown that the proposed algorithm can reduce the drift of target tracking and improve the target tracking performance.


## 3. Scale-Adaptive CACF with Output Constraints

This section describes the principle of the context-aware correlation filtering (CACF) tracker, analyzes filter updating when the output response is Gaussian distribution, introduces the scale adaptive updating method, and provides the specific implementation steps of the suggested method.


### 3.1. Context-Aware Correlation Filter

The classifier of the traditional correlation filter tracking algorithm is trained by means of ridge regression. Its objective function is(1)minωA0ω−y22+λ1ω22,where A0 is a sample set which is obtained by cyclic sampling the target area, λ1 is the regular term coefficient, y is the expected output, and ω is the correlation filter.The symbolAjj≥0 is a cyclic matrix, and it is constructed with a special matrix F. The Aj and its transposed matrix AjT can be obtained as follows(2)Aj=Fdiaga^jFH,AjT=Fdiaga^j∗FH,where ajj≥0 is the target sample, a^j is its Fourier transform, and a^j∗ is its complex conjugate.The correlation filterω can be solved by equation (1) and equation (2) as(3)ω=A0TA0+λ1I−1A0Ty.According to equation (2) j=0 and equation (3), the Fourier form of correlation filter ω is(4)ω^=a^0∗⊙y^a^0∗⊙a^0+λ1.In order to estimate the target position, the response and the confidence map of sampling data need to be found. Let the input isz, and the output is fz:(5)fz=z⋅ω.Then, the Fourier transform offz is(6)y^=f^z=z^⊙ω^,where y^ is the Fourier form of output response, z^ is the Fourier form of input z, and ⊙ is the element-wise product. In order to further simplify the calculation, we set ω=A0Tα, and the simplified result of equation (3) is(7)α=A0TA0+λ1I−1y.According to equation (2) j=0 and equation (7), the Fourier form of correlation filter α is(8)α^=y^a^0∗⊙a^0+λ1.Becauseω=A0Tα, equations (5) and (6) can be calculated as(9)fz=z⋅A0T⋅α,y^=z^⊙a^0∗⊙α^.The context-aware filter not only takes the target area as a sample, but also collects background information around the target area as an auxiliary sample. A new filter is obtained by ridge regression, and the objective function of context-aware filter is(10)minωA0ω−y22+λ1ω22+λ2∑i=1kAiω22,where A0 is the sample set obtained by cyclic sampling of the target area, Ai is the set of samples gained by cyclic sampling of the context area, y is the expected output, ω is the new filter parameter, and λ1 and λ2 are the regular term coefficients of the tracking system.The cyclic matrix can be constructed with equation (2), and set B and y¯ as(11)B=A0λ2A1⋮λ2Ak,y¯=y0⋮0.Then, the objective functionfω,B from equations (10) and (11) is(12)fω,B=Bω−y¯22+λ1ω22.The solutionω of equation (12) can be computed as(13)ω=BTB+λ1I−1BTy¯.According to equations (11) and (13), the Fourier form of correlation filter ω is(14)ω^=a^0∗⊙y^a^0∗⊙a^0+λ1+λ2∑i=1ka^i∗⊙a^i.Lettingω=BTα, we know from equations (13) and (14) that(15)α=BTB+λ1I−1y¯,(16)α^=a^0∗a^0∗⊙a^0+λ1+λ2∑i=1ka^i∗⊙a^0.By taking equation (15) into equation (5), the following equation can be obtained:(17)fz=z⋅BT⋅α.Equation (18) can be computed according to equation (16) and equation (6):(18)y^=z^⊙a^0∗⊙α^0+λ2∑i=1kz^⊙a^i∗⊙α^i.


### 3.2. Filter Updating with Gaussian Distribution

This section discusses the filter updating when the output is unconstrained and when the output response is constrained as Gaussian distribution.


#### 3.2.1. Filter Updating with Unconstrained Output

Lety be the expected output and fz be the actual output, and then the tracking problem can be expressed as(19)εi=yi−ωTφxi,minω,εεi2,ω≤B,wherexi is the M×N-sized image, φxi is a nonlinear transformation and is recorded as the input, yi is the output, and B is a small constant. Equation (20) is a Lagrangian equation which is transformed by equation (19):(20)ℒp=∑i=1M×Nεi2+∑i=1M×Nβiyi−ωTφx−εi+λω2−B.The expressions of variablesα and ω can be obtained by solving the Lagrangian equation as follows:(21)α=K+λI−1y,ω=∑αiφi.Both the modelxt (where t is the frame index) and the filter αt are generally updated with a fixed coefficient η as(22)x^t=1−ηx^t−1+ηxt,(23)α^t=1−ηα^t−1+ηαt.


#### 3.2.2. Filter Updating with Constrained Output

The focus of this section is to introduce a filter with constraints for output response. Assuming that the target output response is satisfied with Gaussian distribution, the objective function can be expressed as(24)minω,ε∑iεi2st:εi=yi−ωTφi,y^t∼Nμt,σ2,t,ω≤B,where yi is the Gaussian representation of the ith sample, ωT is the correlation filter for the ith frame, φi is a nonlinear transformation for the ith frame, μt and σ2,t are the mean and the variance of the Gaussian model p, and y^t is a variable defined by the Gaussian function.Based on the maximum likelihood theory, the optimal solution of Gaussian distribution is(25)y^t−μt22σt+ln2πσt2.It can be seen from equation (25) that only the change of y^t value can affect the optimal solution of Gaussian distribution, and equation (25) can be simplified as(26)y^t−μt2.Sety^t=ωT,txt (where xt represents the tth sample in this sample set); then,(27)ωT,txt−μt2.In order to simplify the calculation amount of optimization process, the following operations are performed on equation (27). According to the theory of Gaussian distribution, the mean μt and μt+1 are updated as(28)μt=1−ρμt−1+ρωT,tx^t,(29)μt−1=1−ρμt−2+ρωT,t−1x^t−1,where x^t is the appearance of the learning target in the tth frame, iteratively acquired by(30)x^t=1−ρx^t−1+ρxt,where xt is the target in the tth frame.By tacking equation (29) into equation (28), we can obtain(31)μt=1−ρ1−ρμt−2+ρωT,t−1x^t−1+ρωT,tx^t.Adding−ωT,txt to both sides of equation (31) and then multiplying (−1) at both sides of the formula, we can obtain(32)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−ρωT,tx^t+ωT,txt.Equation (33) can be acquired by replacing ωT,txt−xt with ωT,txt−x^t:(33)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−ρωT,tx^t+ωT,tx^t+θ1.By taking equation (30) into equation (31), we can have(34)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−1−ρ2ωT,tx^t−1+ρ1−ρωT,txt+θ1,where x^t can be approximately equal to x^t−1 when there is no significant difference between two adjacent frames, and equation (34) can be approximated as(35)ωT,txt−μt=ρ1−ρωT,t−ωT,t−1x^t−1−1−ρ2ωT,tx^t−1−μt−2+θ2.The simplified eqaution (35) is(36)ωT,txt−μt≤ρ1−ρωT,t−ωT,t−1x^t−1+1−ρ2ωT,tx^t−1−μt−2+θ2≤ρ1−ρωt−ωt−1⋅x^t−1+C.It can be seen from equation (36) that ωT,txt−μt also takes the minimum value when ωt−ωt−1 takes the minimum value. Therefore, as long as the optimal solution of ωt−ωt−1 is computed, the optimal solution of ωT,txt−μt is obtained as well.Because the calculation ofωt−ωt−12 is still somewhat complicate, a new variable βit is introduced for further simplifying the calculation, and according to equation (16), we can obtain by setting λ=λ1+λ2∑i=1ka^i∗⊙a^i(37)2εi=βit,2λω=∑iβitφi.The solution ofωt−ωt−12 is transformed into the solution of βt−βt−12, and a new Lagrangian function can be inferred as(38)ℒpαμt,σ2,t=−14∑iβi2,t−14λ∑i.jβitβjtKij+∑iβityi−s∑iβit−βit−12−λB2.Lettingω=∑iαitφi and then βit=2λαit, equation (38) can be computed as(39)maxα,λ−λ2∑iαi2,t+2λ∑iαityi−λ∑i,jαitαjtKij−4λ2s∑iαit−αit−12.The partial derivative ofλ in equation (39) is(40)λI+4λsI+Kαt=y+4λsαt−1.Then, the Fourier transform of equation (40) is(41)Fαμt,σ2,t=Fy+4λsFαt−1Fk+λ+4λs.The simplification of equation (41) can be(42)Fαμt,σ2,t=Fk+λFk+λ+4λs⊙FyFk+λ+1−Fk+λFk+λ+4λs⊙Fαt−1.Lettingτ=Fk+λ/Fk+λ+4λs and taking τ into equation (42), equation (43) can be obtained as(43)Fαμt,σ2,t=τ⊙Fαt+1−τ⊙Fαt−1,where μt and σ2,t are used to select the samples as shown in equation (44) and τ is a matrix with the same size as Fα.Different from the correlation filter using the threshold to detect the fault condition, this paper considers that the Gaussian prior property can prevent from the drift well. The proposed method uses Gaussian prior to select samples when their response output is Gaussian distribution, and it is satisfied with the following equation:(44)y^t−μtσt<G,where G is the empirical value, y^t is the maximum output response, μt is the mean, and σt is the standard deviation.According to the basic principles of mean and variance, we know thatμt and σ2,t are(45)μt=∑i=1ty^it,(46)σ2,t=∑i=1ty^i−μt2t,where y^i is the maximum output response of the ith frame.


### 3.3. Scale Updating with Maximum Posterior Probability Distribution

At present, many scale updating methods are based on the likelihood principle. In the tracking detection stage, the maximum possibility of each scale is used to find the optimal scale in the scale pool. We can set the size of scale template asST=Sx,Sy and its zoom pool as S=t1t2…tk at first, and then we can use k windows with the tistti∈S size to find the target and use bilinear interpolation to adjust the sample size to the fixed template size ST by assuming that the target window is st in the original image. The response is calculated as follows:(47)fzti=Kxz⊙α,where zti is the sample patch with the tist size, Kxz is the correlation operation between the sample z and template x, and α is the filter.According to equation (47), the maximum response is(48)argmaxiF−1f^zti.After a set of data is obtained using the response functionf^zti, then the maximum function is used to find its maximum scalar. Because the target motion is hidden in the response graph, it is necessary to use ti to adjust the final displacement to get the real motion deviation.Although the maximum likelihood method has made some achievements in dealing with the problem of the target scale, it fails to solve the problem of the complex target scale. In order to solve the scale problem of the complex target, this paper adopts the maximum posterior probability distribution method instead of the maximum likelihood method, and the details are shown as(49)maxiPsiy=PysiPsi,(50)ys=argmaxiF−1f^ztiPsi,where si is the ith scale, y is the output in the current frame, Pysi is the likelihood which can be calculated from the maximum response peak value at the ith scale, ys is the optimal scale response of the current frame, and Psi is a possibility to follow a Gaussian distribution, which can be obtained by centering around the previous scale and the standard deviation which is set in the experiment.


### 3.4. Steps of the Proposed Algorithm

The detailed steps of the proposed algorithm are summarized in Algorithm1 based on the results of studies in Sections 3.1, 3.2, and 3.3.Algorithm 1: Scale adaptive context-aware correlation filter with output constraints for target tracking.
Inputs:b0: target location and size of the first framex^0: the target model of the first frameα^0: the correlation filter of the first frameλ1,λ2: the regular term coefficients of the tracking systemOutputs:bn−1: target location and size of the previous framex^n: the target model of the current frameα^n: the correlation filter of the current framePsi: maximum probability of the ith scaleyscale: optimal scale response of the current frameyn: the maximum correlation filter response value of the current framePn: target location of the current framePreprocessing position and model(1)
Initialize the bounding box of the targetb0=x0,y0,w,h(2)
When the frame numbern is less than 11, calculate the best mean and the variance of Gaussian constraint with equations (45) and (46)(3)
Get a search window based onbn−1n≥1, calculate the maximum correlation response y^ according to equation (16), and then mark the maximum correlation filter response value as yn, and the target position Pn is obtained from yn(4)
Continue step (5) ifn<11, and go to step 8 if n≥11(5)
Update the target modelx^n from equation (22)(6)
Update the correlation filterα^n with equation (23)(7)
Return to step (3)Model updating(8)
Update the target modelx^n using equation (22)Scale updating(9)
Calculate the responsePsi according to equations (47) and (48)(10)
The maximum posterior probability is calculated by equation (47), and the optimal scale response yscale is obtained from equation (50)Filter updatingify^−μ/σ<G(11)
Update the correlation filterα^n according to equation (43)else(12)
Update the correlation filterα^n using equation (23)end(13)
Repeat step (3)Until End of the video sequenceEnd


## 3.1. Context-Aware Correlation Filter

The classifier of the traditional correlation filter tracking algorithm is trained by means of ridge regression. Its objective function is(1)minωA0ω−y22+λ1ω22,where A0 is a sample set which is obtained by cyclic sampling the target area, λ1 is the regular term coefficient, y is the expected output, and ω is the correlation filter.The symbolAjj≥0 is a cyclic matrix, and it is constructed with a special matrix F. The Aj and its transposed matrix AjT can be obtained as follows(2)Aj=Fdiaga^jFH,AjT=Fdiaga^j∗FH,where ajj≥0 is the target sample, a^j is its Fourier transform, and a^j∗ is its complex conjugate.The correlation filterω can be solved by equation (1) and equation (2) as(3)ω=A0TA0+λ1I−1A0Ty.According to equation (2) j=0 and equation (3), the Fourier form of correlation filter ω is(4)ω^=a^0∗⊙y^a^0∗⊙a^0+λ1.In order to estimate the target position, the response and the confidence map of sampling data need to be found. Let the input isz, and the output is fz:(5)fz=z⋅ω.Then, the Fourier transform offz is(6)y^=f^z=z^⊙ω^,where y^ is the Fourier form of output response, z^ is the Fourier form of input z, and ⊙ is the element-wise product. In order to further simplify the calculation, we set ω=A0Tα, and the simplified result of equation (3) is(7)α=A0TA0+λ1I−1y.According to equation (2) j=0 and equation (7), the Fourier form of correlation filter α is(8)α^=y^a^0∗⊙a^0+λ1.Becauseω=A0Tα, equations (5) and (6) can be calculated as(9)fz=z⋅A0T⋅α,y^=z^⊙a^0∗⊙α^.The context-aware filter not only takes the target area as a sample, but also collects background information around the target area as an auxiliary sample. A new filter is obtained by ridge regression, and the objective function of context-aware filter is(10)minωA0ω−y22+λ1ω22+λ2∑i=1kAiω22,where A0 is the sample set obtained by cyclic sampling of the target area, Ai is the set of samples gained by cyclic sampling of the context area, y is the expected output, ω is the new filter parameter, and λ1 and λ2 are the regular term coefficients of the tracking system.The cyclic matrix can be constructed with equation (2), and set B and y¯ as(11)B=A0λ2A1⋮λ2Ak,y¯=y0⋮0.Then, the objective functionfω,B from equations (10) and (11) is(12)fω,B=Bω−y¯22+λ1ω22.The solutionω of equation (12) can be computed as(13)ω=BTB+λ1I−1BTy¯.According to equations (11) and (13), the Fourier form of correlation filter ω is(14)ω^=a^0∗⊙y^a^0∗⊙a^0+λ1+λ2∑i=1ka^i∗⊙a^i.Lettingω=BTα, we know from equations (13) and (14) that(15)α=BTB+λ1I−1y¯,(16)α^=a^0∗a^0∗⊙a^0+λ1+λ2∑i=1ka^i∗⊙a^0.By taking equation (15) into equation (5), the following equation can be obtained:(17)fz=z⋅BT⋅α.Equation (18) can be computed according to equation (16) and equation (6):(18)y^=z^⊙a^0∗⊙α^0+λ2∑i=1kz^⊙a^i∗⊙α^i.


## 3.2. Filter Updating with Gaussian Distribution

This section discusses the filter updating when the output is unconstrained and when the output response is constrained as Gaussian distribution.


### 3.2.1. Filter Updating with Unconstrained Output

Lety be the expected output and fz be the actual output, and then the tracking problem can be expressed as(19)εi=yi−ωTφxi,minω,εεi2,ω≤B,wherexi is the M×N-sized image, φxi is a nonlinear transformation and is recorded as the input, yi is the output, and B is a small constant. Equation (20) is a Lagrangian equation which is transformed by equation (19):(20)ℒp=∑i=1M×Nεi2+∑i=1M×Nβiyi−ωTφx−εi+λω2−B.The expressions of variablesα and ω can be obtained by solving the Lagrangian equation as follows:(21)α=K+λI−1y,ω=∑αiφi.Both the modelxt (where t is the frame index) and the filter αt are generally updated with a fixed coefficient η as(22)x^t=1−ηx^t−1+ηxt,(23)α^t=1−ηα^t−1+ηαt.


### 3.2.2. Filter Updating with Constrained Output

The focus of this section is to introduce a filter with constraints for output response. Assuming that the target output response is satisfied with Gaussian distribution, the objective function can be expressed as(24)minω,ε∑iεi2st:εi=yi−ωTφi,y^t∼Nμt,σ2,t,ω≤B,where yi is the Gaussian representation of the ith sample, ωT is the correlation filter for the ith frame, φi is a nonlinear transformation for the ith frame, μt and σ2,t are the mean and the variance of the Gaussian model p, and y^t is a variable defined by the Gaussian function.Based on the maximum likelihood theory, the optimal solution of Gaussian distribution is(25)y^t−μt22σt+ln2πσt2.It can be seen from equation (25) that only the change of y^t value can affect the optimal solution of Gaussian distribution, and equation (25) can be simplified as(26)y^t−μt2.Sety^t=ωT,txt (where xt represents the tth sample in this sample set); then,(27)ωT,txt−μt2.In order to simplify the calculation amount of optimization process, the following operations are performed on equation (27). According to the theory of Gaussian distribution, the mean μt and μt+1 are updated as(28)μt=1−ρμt−1+ρωT,tx^t,(29)μt−1=1−ρμt−2+ρωT,t−1x^t−1,where x^t is the appearance of the learning target in the tth frame, iteratively acquired by(30)x^t=1−ρx^t−1+ρxt,where xt is the target in the tth frame.By tacking equation (29) into equation (28), we can obtain(31)μt=1−ρ1−ρμt−2+ρωT,t−1x^t−1+ρωT,tx^t.Adding−ωT,txt to both sides of equation (31) and then multiplying (−1) at both sides of the formula, we can obtain(32)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−ρωT,tx^t+ωT,txt.Equation (33) can be acquired by replacing ωT,txt−xt with ωT,txt−x^t:(33)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−ρωT,tx^t+ωT,tx^t+θ1.By taking equation (30) into equation (31), we can have(34)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−1−ρ2ωT,tx^t−1+ρ1−ρωT,txt+θ1,where x^t can be approximately equal to x^t−1 when there is no significant difference between two adjacent frames, and equation (34) can be approximated as(35)ωT,txt−μt=ρ1−ρωT,t−ωT,t−1x^t−1−1−ρ2ωT,tx^t−1−μt−2+θ2.The simplified eqaution (35) is(36)ωT,txt−μt≤ρ1−ρωT,t−ωT,t−1x^t−1+1−ρ2ωT,tx^t−1−μt−2+θ2≤ρ1−ρωt−ωt−1⋅x^t−1+C.It can be seen from equation (36) that ωT,txt−μt also takes the minimum value when ωt−ωt−1 takes the minimum value. Therefore, as long as the optimal solution of ωt−ωt−1 is computed, the optimal solution of ωT,txt−μt is obtained as well.Because the calculation ofωt−ωt−12 is still somewhat complicate, a new variable βit is introduced for further simplifying the calculation, and according to equation (16), we can obtain by setting λ=λ1+λ2∑i=1ka^i∗⊙a^i(37)2εi=βit,2λω=∑iβitφi.The solution ofωt−ωt−12 is transformed into the solution of βt−βt−12, and a new Lagrangian function can be inferred as(38)ℒpαμt,σ2,t=−14∑iβi2,t−14λ∑i.jβitβjtKij+∑iβityi−s∑iβit−βit−12−λB2.Lettingω=∑iαitφi and then βit=2λαit, equation (38) can be computed as(39)maxα,λ−λ2∑iαi2,t+2λ∑iαityi−λ∑i,jαitαjtKij−4λ2s∑iαit−αit−12.The partial derivative ofλ in equation (39) is(40)λI+4λsI+Kαt=y+4λsαt−1.Then, the Fourier transform of equation (40) is(41)Fαμt,σ2,t=Fy+4λsFαt−1Fk+λ+4λs.The simplification of equation (41) can be(42)Fαμt,σ2,t=Fk+λFk+λ+4λs⊙FyFk+λ+1−Fk+λFk+λ+4λs⊙Fαt−1.Lettingτ=Fk+λ/Fk+λ+4λs and taking τ into equation (42), equation (43) can be obtained as(43)Fαμt,σ2,t=τ⊙Fαt+1−τ⊙Fαt−1,where μt and σ2,t are used to select the samples as shown in equation (44) and τ is a matrix with the same size as Fα.Different from the correlation filter using the threshold to detect the fault condition, this paper considers that the Gaussian prior property can prevent from the drift well. The proposed method uses Gaussian prior to select samples when their response output is Gaussian distribution, and it is satisfied with the following equation:(44)y^t−μtσt<G,where G is the empirical value, y^t is the maximum output response, μt is the mean, and σt is the standard deviation.According to the basic principles of mean and variance, we know thatμt and σ2,t are(45)μt=∑i=1ty^it,(46)σ2,t=∑i=1ty^i−μt2t,where y^i is the maximum output response of the ith frame.


## 3.2.1. Filter Updating with Unconstrained Output

Lety be the expected output and fz be the actual output, and then the tracking problem can be expressed as(19)εi=yi−ωTφxi,minω,εεi2,ω≤B,wherexi is the M×N-sized image, φxi is a nonlinear transformation and is recorded as the input, yi is the output, and B is a small constant. Equation (20) is a Lagrangian equation which is transformed by equation (19):(20)ℒp=∑i=1M×Nεi2+∑i=1M×Nβiyi−ωTφx−εi+λω2−B.The expressions of variablesα and ω can be obtained by solving the Lagrangian equation as follows:(21)α=K+λI−1y,ω=∑αiφi.Both the modelxt (where t is the frame index) and the filter αt are generally updated with a fixed coefficient η as(22)x^t=1−ηx^t−1+ηxt,(23)α^t=1−ηα^t−1+ηαt.


## 3.2.2. Filter Updating with Constrained Output

The focus of this section is to introduce a filter with constraints for output response. Assuming that the target output response is satisfied with Gaussian distribution, the objective function can be expressed as(24)minω,ε∑iεi2st:εi=yi−ωTφi,y^t∼Nμt,σ2,t,ω≤B,where yi is the Gaussian representation of the ith sample, ωT is the correlation filter for the ith frame, φi is a nonlinear transformation for the ith frame, μt and σ2,t are the mean and the variance of the Gaussian model p, and y^t is a variable defined by the Gaussian function.Based on the maximum likelihood theory, the optimal solution of Gaussian distribution is(25)y^t−μt22σt+ln2πσt2.It can be seen from equation (25) that only the change of y^t value can affect the optimal solution of Gaussian distribution, and equation (25) can be simplified as(26)y^t−μt2.Sety^t=ωT,txt (where xt represents the tth sample in this sample set); then,(27)ωT,txt−μt2.In order to simplify the calculation amount of optimization process, the following operations are performed on equation (27). According to the theory of Gaussian distribution, the mean μt and μt+1 are updated as(28)μt=1−ρμt−1+ρωT,tx^t,(29)μt−1=1−ρμt−2+ρωT,t−1x^t−1,where x^t is the appearance of the learning target in the tth frame, iteratively acquired by(30)x^t=1−ρx^t−1+ρxt,where xt is the target in the tth frame.By tacking equation (29) into equation (28), we can obtain(31)μt=1−ρ1−ρμt−2+ρωT,t−1x^t−1+ρωT,tx^t.Adding−ωT,txt to both sides of equation (31) and then multiplying (−1) at both sides of the formula, we can obtain(32)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−ρωT,tx^t+ωT,txt.Equation (33) can be acquired by replacing ωT,txt−xt with ωT,txt−x^t:(33)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−ρωT,tx^t+ωT,tx^t+θ1.By taking equation (30) into equation (31), we can have(34)ωT,txt−μt=−ρ1−ρωT,t−1x^t−1−1−ρ2μt−2−1−ρ2ωT,tx^t−1+ρ1−ρωT,txt+θ1,where x^t can be approximately equal to x^t−1 when there is no significant difference between two adjacent frames, and equation (34) can be approximated as(35)ωT,txt−μt=ρ1−ρωT,t−ωT,t−1x^t−1−1−ρ2ωT,tx^t−1−μt−2+θ2.The simplified eqaution (35) is(36)ωT,txt−μt≤ρ1−ρωT,t−ωT,t−1x^t−1+1−ρ2ωT,tx^t−1−μt−2+θ2≤ρ1−ρωt−ωt−1⋅x^t−1+C.It can be seen from equation (36) that ωT,txt−μt also takes the minimum value when ωt−ωt−1 takes the minimum value. Therefore, as long as the optimal solution of ωt−ωt−1 is computed, the optimal solution of ωT,txt−μt is obtained as well.Because the calculation ofωt−ωt−12 is still somewhat complicate, a new variable βit is introduced for further simplifying the calculation, and according to equation (16), we can obtain by setting λ=λ1+λ2∑i=1ka^i∗⊙a^i(37)2εi=βit,2λω=∑iβitφi.The solution ofωt−ωt−12 is transformed into the solution of βt−βt−12, and a new Lagrangian function can be inferred as(38)ℒpαμt,σ2,t=−14∑iβi2,t−14λ∑i.jβitβjtKij+∑iβityi−s∑iβit−βit−12−λB2.Lettingω=∑iαitφi and then βit=2λαit, equation (38) can be computed as(39)maxα,λ−λ2∑iαi2,t+2λ∑iαityi−λ∑i,jαitαjtKij−4λ2s∑iαit−αit−12.The partial derivative ofλ in equation (39) is(40)λI+4λsI+Kαt=y+4λsαt−1.Then, the Fourier transform of equation (40) is(41)Fαμt,σ2,t=Fy+4λsFαt−1Fk+λ+4λs.The simplification of equation (41) can be(42)Fαμt,σ2,t=Fk+λFk+λ+4λs⊙FyFk+λ+1−Fk+λFk+λ+4λs⊙Fαt−1.Lettingτ=Fk+λ/Fk+λ+4λs and taking τ into equation (42), equation (43) can be obtained as(43)Fαμt,σ2,t=τ⊙Fαt+1−τ⊙Fαt−1,where μt and σ2,t are used to select the samples as shown in equation (44) and τ is a matrix with the same size as Fα.Different from the correlation filter using the threshold to detect the fault condition, this paper considers that the Gaussian prior property can prevent from the drift well. The proposed method uses Gaussian prior to select samples when their response output is Gaussian distribution, and it is satisfied with the following equation:(44)y^t−μtσt<G,where G is the empirical value, y^t is the maximum output response, μt is the mean, and σt is the standard deviation.According to the basic principles of mean and variance, we know thatμt and σ2,t are(45)μt=∑i=1ty^it,(46)σ2,t=∑i=1ty^i−μt2t,where y^i is the maximum output response of the ith frame.


## 3.3. Scale Updating with Maximum Posterior Probability Distribution

At present, many scale updating methods are based on the likelihood principle. In the tracking detection stage, the maximum possibility of each scale is used to find the optimal scale in the scale pool. We can set the size of scale template asST=Sx,Sy and its zoom pool as S=t1t2…tk at first, and then we can use k windows with the tistti∈S size to find the target and use bilinear interpolation to adjust the sample size to the fixed template size ST by assuming that the target window is st in the original image. The response is calculated as follows:(47)fzti=Kxz⊙α,where zti is the sample patch with the tist size, Kxz is the correlation operation between the sample z and template x, and α is the filter.According to equation (47), the maximum response is(48)argmaxiF−1f^zti.After a set of data is obtained using the response functionf^zti, then the maximum function is used to find its maximum scalar. Because the target motion is hidden in the response graph, it is necessary to use ti to adjust the final displacement to get the real motion deviation.Although the maximum likelihood method has made some achievements in dealing with the problem of the target scale, it fails to solve the problem of the complex target scale. In order to solve the scale problem of the complex target, this paper adopts the maximum posterior probability distribution method instead of the maximum likelihood method, and the details are shown as(49)maxiPsiy=PysiPsi,(50)ys=argmaxiF−1f^ztiPsi,where si is the ith scale, y is the output in the current frame, Pysi is the likelihood which can be calculated from the maximum response peak value at the ith scale, ys is the optimal scale response of the current frame, and Psi is a possibility to follow a Gaussian distribution, which can be obtained by centering around the previous scale and the standard deviation which is set in the experiment.


## 3.4. Steps of the Proposed Algorithm

The detailed steps of the proposed algorithm are summarized in Algorithm1 based on the results of studies in Sections 3.1, 3.2, and 3.3.Algorithm 1: Scale adaptive context-aware correlation filter with output constraints for target tracking.
Inputs:b0: target location and size of the first framex^0: the target model of the first frameα^0: the correlation filter of the first frameλ1,λ2: the regular term coefficients of the tracking systemOutputs:bn−1: target location and size of the previous framex^n: the target model of the current frameα^n: the correlation filter of the current framePsi: maximum probability of the ith scaleyscale: optimal scale response of the current frameyn: the maximum correlation filter response value of the current framePn: target location of the current framePreprocessing position and model(1)
Initialize the bounding box of the targetb0=x0,y0,w,h(2)
When the frame numbern is less than 11, calculate the best mean and the variance of Gaussian constraint with equations (45) and (46)(3)
Get a search window based onbn−1n≥1, calculate the maximum correlation response y^ according to equation (16), and then mark the maximum correlation filter response value as yn, and the target position Pn is obtained from yn(4)
Continue step (5) ifn<11, and go to step 8 if n≥11(5)
Update the target modelx^n from equation (22)(6)
Update the correlation filterα^n with equation (23)(7)
Return to step (3)Model updating(8)
Update the target modelx^n using equation (22)Scale updating(9)
Calculate the responsePsi according to equations (47) and (48)(10)
The maximum posterior probability is calculated by equation (47), and the optimal scale response yscale is obtained from equation (50)Filter updatingify^−μ/σ<G(11)
Update the correlation filterα^n according to equation (43)else(12)
Update the correlation filterα^n using equation (23)end(13)
Repeat step (3)Until End of the video sequenceEnd


## 4. Experiments

The proposed OURS tracker in this paper is compared with some excellent trackers in the standard test set OTB-50 and OTB-100. In the OTB dataset, each sequence has 11 attributes including illumination variation (IV), out-of-plane rotation (OPR), scale variation (SV), occlusion (OCC), deformation (DEF), motion blur (MB), fast motion (FM), in-plane rotation (IPR), out-of-view (OV), background clutter (BC), and low resolution (LR) that represent challenging aspects of target tracking. The comparison algorithms include CSK [28], CACF [25], KCF [32], and Staple [34]. The precision curve is the percentage of estimated position which is within the given threshold of the ground truth, and the success curve is the coincidence rate between the predicted target position and the actual target position. The precision curve and the success curve are taken as evaluation criteria in our work. When the coincidence rate of this frame is greater than the given threshold, it is recorded as successful tracking, otherwise it fails. In this study, the threshold of the precision curve is set to 20 px, and the threshold of the success curve is set to 0.5.


### 4.1. Parameter Setup

This experiment tests the effect of parameters on the OURS tracker. For example, an experiment is completed based on a subset of OTB with differents values as shown in Figure 2, and the precision is the highest when s is 1050. In order to choose the best constraint G in equation (43), this study tested the different values in Figure 3, the precision is not changed much when G is varied from 1.0 to 1.5, and the precision is the highest when G is 1.7, so we take G is 1.7 in this work. In the relevant filtering part, the feature is taken as a combination of the HOG feature and color feature, and the kernel function is Gaussian kernel function. Let λ1=10−4 and λ2=0.4. Depending on experience, the standard deviation of Gaussian kernel function is σ2=0.5. The selection of other parameters in the proposed OURS tracker is basically consistent with the CACF tracker [27].Figure 2
The evaluation ofS based on precision.Figure 3
The evaluation ofG based on precision.


### 4.2. Experiments on the Full Dataset OTB-50

This work is done by evaluating OPE and evaluates the tracker by running the tracker on 51 benchmark video sequences, and the actual ground position is initialized in the first frame and the average accuracy is derived. The precision and the success rate of various trackers are tested in the standard test set OTB-2013, and their test results are shown in Figure4. The proposed OURS tracker achieves a score of 0.845 for the precision low threshold value of 20, and it achieves a score of 0.774 for the success rate above the overlap threshold value of 0.5; the OURS tracker achieves the best performance in these trackers. The comparison of the OURS tracker and CACF tracker is summarized in Table 1. The OURS tracker gets higher precision than the CACF tracker when dealing with sequences containing illumination variation (IV), scale variation (SV), out-of-plane rotation (OPR), occlusion (OCC), deformation (DEF), in-plane rotation (IPR), motion blur (MB), and fast motion (FM) attributes. The running speed of the OURS tracker and other trackers is shown in Figure 5. It can be seen from Figure 5 that though the running speed of the CSK tracker is the highest and the running speed of the OURS tracker is the lowest, the running speed of the OURS tracker is only 13 frames slower than the CACF tracker. Comprehensive analysis from the high performance and running speed of the OURS tracker shows that the OURS tracker still has high application value.Figure 4
The average precision and success plot of 5 trackers (OTB-50). (a) Precision plots of OPE. (b) Success plots of OPE.
(a)(b)Table 1
Performance (mean precision (20 px)) comparisons of the proposed method with the CACF tracker.
MethodOverallIVOPRSVOCCDEFMBFMIPROVBCLROURS (%)84.581.384.879.682.982.474.673.979.775.778.651.5CACF (%)80.377.378.374.079.578.371.768.473.976.577.854.7Figure 5
Speed comparisons of the proposed method with other trackers (OTB-50).


### 4.3. Experiments with Various Challenging Sequence Attributes on OTB-50

The attributes of standard test set annotations describe the challenges that the tracker faces within each video sequence. This test set allows the researcher to judge and characterize the behavior of the tracker without having to analyze each video sequence. In Figure6, this experiment counts the precision of each attribute, and it is easy to see that the OURS tracker works the best properties. The success rate of each attribute is shown in Table 2, and the maximum value of success rates is represented by a bold font. It can be seen from Table 2 that the OURS tracker has the highest success rate when dealing with illumination variation (IV), out-of-plane rotation (OPR), scale variation (SV), occlusion (OCC), deformation (DEF), motion blur (MB), and fast motion (FM). Although the success rate of Staple tracker is highest when dealing with in-plane rotation (IPR) and the success rate of CACF tracker has the highest success rate when dealing with out-of-view (OV), background clutter (BC), and low resolution (LR), the OURS tracker has also achieved the near-optimal performance in these four attributes. In a word, the OURS tracker achieves better performance in most cases.Figure 6
Detailed results on each attribute (OTB-50).Precision plots of (a) OPE-fast motion (17). (b) OPE-occlusion (29). (c) OPE-deformation (19). (d) OPE-scale variation (28). (e) OPE-background clutter (21). (f) OPE-out-of-plane rotation (39). (g) OPE-in-plane rotation (31). (h) OPE-illumination variation (25). (i) OPE-motion blur (12). (j) OPE-out-of-view (6). (k) OPE-low resolution (4).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)Table 2
Success rate comparisons of the tracker on different attributes.
TrackersIVOPRSVOCCDEFMBFMIPROVBCLROURS73.775.668.976.480.471.469.370.976.172.151.2Staple69.270.665.473.277.462.860.571.458.669.849.9KCF58.160.847.961.867.159.555.761.565.067.235.7CACF71.972.065.974.378.569.065.567.876.875.454.2CSK38.843.935.240.037.033.638.045.741.049.139.7


### 4.4. Experiments on the Full Dataset OTB-100

To better verify the superiority of OURS tracker, the accuracy and success rate of the OURS tracker and other trackers are tested on the standard test set OTB-100, and the test results are shown in Figure7. According to Figure 7, we know that the accuracy scores of OURS tracker and CACF tracker are 0.827 and 0.793 when the threshold value is lower than 20, so the tracking accuracy of OURS tracker is 0.034 higher than CACF tracker. The success rate scores of OURS tracker and CACF tracker are 0.718 and 0.699 when the overlap threshold value is more than 0.5, so the tracking success rate of OURS tracker is 0.019 higher than that of CACF tracker. In order to better highlight the value of OURS tracker, the comprehensive performance of five trackers is shown in Table 3. It can be seen from Table 3 that the tracking accuracy and tracking success rate of OURS tracker is the highest among the 5 trackers. Although the running speed of the OURS tracker is lower, it is only 14.13 frames slower than the running speed of CACF tracker. In the case of low running speed requirement, the OURS tracker has better tracker performance than other trackers.Figure 7
The average precision and success plot of 5 trackers (OTB-100). (a) Precision plots of OPE. (b) Success plots of OPE.
(a)(b)Table 3
Comprehensive tracking performance of each tracker.
MethodOURSCACFStapleKCFCSKPrecision0.8270.7930.7840.6960.518Success0.7180.6990.6950.5510.411FPS21.6535.58166.92214.12262.54


### 4.5. Experiments with Various Challenging Sequence Attributes on OTB-100

To verify the effectiveness of the OURS tracker, the OURS tracker is again compared with other trackers on the standard test set OTB-100. It can be seen from Figure8 that the OURS tracker has achieved the best tracking performance when tracking objects with occlusion, geometric deformation, and fast movement. By solving the tracking performance of these main target attributes, the problem of tracking target drift is greatly alleviated. The OURS tracker has the best tracking performance on most attributes. In addition, we know from Figure 9 that the success rate of OURS tracker has obvious advantages compared with other trackers on most attributes. In summary, the OURS tracker achieves the best tracking performance compared with other trackers in almost all attributes.Figure 8
Detailed results on each attribute (OTB-100). Precision plots of (a) OPE-occlusion (48). (b) OPE-fast motion (39). (c) OPE-background clutter (31). (d) OPE-out-of-plane rotation (63). (e) OPE-in-plane rotation (51). (f) OPE-low resolution (9). (g) OPE-illumination variation (37). (h) OPE-occlusion (48). (i) OPE-motion blur (29). (j) OPE-out of view (14). (k) OPE-scale variation (63).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)Figure 9
Comparison of tracker success rates (OTB-100).


### 4.6. Experiments under Different Video Sequences

In order to more accurately describe the performance of the OURS tracker, this paper selects nine video sequences and tests their precision. The precision of nine video sequences in this experiment is shown in Table4, and the best results are reported in bold. According to the test results in Table 4, the OURS tracker has the highest precision when tracking Bolt, Coke, Couple, Jumping, Freeman1, Freeman4, Jogging, and Girl video sequences, and it also achieved near-highest precision when tracking Football video sequences. The OURS tracker performs better than other four trackers.Table 4
Performance (mean precision (20 px)) comparisons of different trackers in some video sequences.
TrackersBoltCokeCoupleJumpingFreeman1Freeman4JoggingGirlFootballOURS10096.789.783.595.975.597.410081.7Staple10091.834.944.937.112.536.610074.9KCF67.182.826.736.139.653.023.586.276.8CACF50.792.155.749.538.049.536.810078.8CSK23.591.210.620.140.838.954.253.683.4This experiment provides a qualitative comparison of OURS tracker with the existed tracker in order to verify the stability of OURS tracker. These video sequences constitute different challenging situations including illumination variation (IV), out-of-plane rotation (OPR), scale variation (SV), in-plane rotation (IPR), background clutter (BC), occlusion (OCC), deformation (DEF), fast motion (FM), and motion blur (MB). The actual tracking results of OURS tracker, CACF tracker, and KCF tracker in four video sequences are shown in Figure10. Figure 10(a) shows that the KCF tracker cannot stably track video sequences that contain scale variation attribute, but the OURS tracker and the CACF tracker can consistently track the Shaking video sequence. Figure 10(b) shows that the CACF tracker cannot successfully track the Bolt sequence when the video sequence contains in-plane rotation attributes, but the OURS tracker and the KCF tracker can handle these issues very well. Figure 10(c) shows that only the OURS tracker can stably track Freeman video sequences and neither the KCF tracker nor CACF tracker can successfully track the sequence. Figure 10(d) shows that only the OURS tracker can successfully track the Jumping video sequence when the video contains fast motion and motion blur properties. In summary, the OURS tracker has excellent stability.Figure 10
A visualization of the tracking results of OURS tracker and some visual trackers (CACF [25]and KCF [32]) on four benchmark sequences. (a) Shaking (IV, OPR, SV, and BC). (b) Bolt (OPR, OCC, DEF, and IPR). (c) Freeman1 (OPR, SV, and IPR). (d) Jumping (MB and FM).
(a)(b)(c)(d)


## 4.1. Parameter Setup

This experiment tests the effect of parameters on the OURS tracker. For example, an experiment is completed based on a subset of OTB with differents values as shown in Figure 2, and the precision is the highest when s is 1050. In order to choose the best constraint G in equation (43), this study tested the different values in Figure 3, the precision is not changed much when G is varied from 1.0 to 1.5, and the precision is the highest when G is 1.7, so we take G is 1.7 in this work. In the relevant filtering part, the feature is taken as a combination of the HOG feature and color feature, and the kernel function is Gaussian kernel function. Let λ1=10−4 and λ2=0.4. Depending on experience, the standard deviation of Gaussian kernel function is σ2=0.5. The selection of other parameters in the proposed OURS tracker is basically consistent with the CACF tracker [27].Figure 2
The evaluation ofS based on precision.Figure 3
The evaluation ofG based on precision.


## 4.2. Experiments on the Full Dataset OTB-50

This work is done by evaluating OPE and evaluates the tracker by running the tracker on 51 benchmark video sequences, and the actual ground position is initialized in the first frame and the average accuracy is derived. The precision and the success rate of various trackers are tested in the standard test set OTB-2013, and their test results are shown in Figure4. The proposed OURS tracker achieves a score of 0.845 for the precision low threshold value of 20, and it achieves a score of 0.774 for the success rate above the overlap threshold value of 0.5; the OURS tracker achieves the best performance in these trackers. The comparison of the OURS tracker and CACF tracker is summarized in Table 1. The OURS tracker gets higher precision than the CACF tracker when dealing with sequences containing illumination variation (IV), scale variation (SV), out-of-plane rotation (OPR), occlusion (OCC), deformation (DEF), in-plane rotation (IPR), motion blur (MB), and fast motion (FM) attributes. The running speed of the OURS tracker and other trackers is shown in Figure 5. It can be seen from Figure 5 that though the running speed of the CSK tracker is the highest and the running speed of the OURS tracker is the lowest, the running speed of the OURS tracker is only 13 frames slower than the CACF tracker. Comprehensive analysis from the high performance and running speed of the OURS tracker shows that the OURS tracker still has high application value.Figure 4
The average precision and success plot of 5 trackers (OTB-50). (a) Precision plots of OPE. (b) Success plots of OPE.
(a)(b)Table 1
Performance (mean precision (20 px)) comparisons of the proposed method with the CACF tracker.
MethodOverallIVOPRSVOCCDEFMBFMIPROVBCLROURS (%)84.581.384.879.682.982.474.673.979.775.778.651.5CACF (%)80.377.378.374.079.578.371.768.473.976.577.854.7Figure 5
Speed comparisons of the proposed method with other trackers (OTB-50).


## 4.3. Experiments with Various Challenging Sequence Attributes on OTB-50

The attributes of standard test set annotations describe the challenges that the tracker faces within each video sequence. This test set allows the researcher to judge and characterize the behavior of the tracker without having to analyze each video sequence. In Figure6, this experiment counts the precision of each attribute, and it is easy to see that the OURS tracker works the best properties. The success rate of each attribute is shown in Table 2, and the maximum value of success rates is represented by a bold font. It can be seen from Table 2 that the OURS tracker has the highest success rate when dealing with illumination variation (IV), out-of-plane rotation (OPR), scale variation (SV), occlusion (OCC), deformation (DEF), motion blur (MB), and fast motion (FM). Although the success rate of Staple tracker is highest when dealing with in-plane rotation (IPR) and the success rate of CACF tracker has the highest success rate when dealing with out-of-view (OV), background clutter (BC), and low resolution (LR), the OURS tracker has also achieved the near-optimal performance in these four attributes. In a word, the OURS tracker achieves better performance in most cases.Figure 6
Detailed results on each attribute (OTB-50).Precision plots of (a) OPE-fast motion (17). (b) OPE-occlusion (29). (c) OPE-deformation (19). (d) OPE-scale variation (28). (e) OPE-background clutter (21). (f) OPE-out-of-plane rotation (39). (g) OPE-in-plane rotation (31). (h) OPE-illumination variation (25). (i) OPE-motion blur (12). (j) OPE-out-of-view (6). (k) OPE-low resolution (4).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)Table 2
Success rate comparisons of the tracker on different attributes.
TrackersIVOPRSVOCCDEFMBFMIPROVBCLROURS73.775.668.976.480.471.469.370.976.172.151.2Staple69.270.665.473.277.462.860.571.458.669.849.9KCF58.160.847.961.867.159.555.761.565.067.235.7CACF71.972.065.974.378.569.065.567.876.875.454.2CSK38.843.935.240.037.033.638.045.741.049.139.7


## 4.4. Experiments on the Full Dataset OTB-100

To better verify the superiority of OURS tracker, the accuracy and success rate of the OURS tracker and other trackers are tested on the standard test set OTB-100, and the test results are shown in Figure7. According to Figure 7, we know that the accuracy scores of OURS tracker and CACF tracker are 0.827 and 0.793 when the threshold value is lower than 20, so the tracking accuracy of OURS tracker is 0.034 higher than CACF tracker. The success rate scores of OURS tracker and CACF tracker are 0.718 and 0.699 when the overlap threshold value is more than 0.5, so the tracking success rate of OURS tracker is 0.019 higher than that of CACF tracker. In order to better highlight the value of OURS tracker, the comprehensive performance of five trackers is shown in Table 3. It can be seen from Table 3 that the tracking accuracy and tracking success rate of OURS tracker is the highest among the 5 trackers. Although the running speed of the OURS tracker is lower, it is only 14.13 frames slower than the running speed of CACF tracker. In the case of low running speed requirement, the OURS tracker has better tracker performance than other trackers.Figure 7
The average precision and success plot of 5 trackers (OTB-100). (a) Precision plots of OPE. (b) Success plots of OPE.
(a)(b)Table 3
Comprehensive tracking performance of each tracker.
MethodOURSCACFStapleKCFCSKPrecision0.8270.7930.7840.6960.518Success0.7180.6990.6950.5510.411FPS21.6535.58166.92214.12262.54


## 4.5. Experiments with Various Challenging Sequence Attributes on OTB-100

To verify the effectiveness of the OURS tracker, the OURS tracker is again compared with other trackers on the standard test set OTB-100. It can be seen from Figure8 that the OURS tracker has achieved the best tracking performance when tracking objects with occlusion, geometric deformation, and fast movement. By solving the tracking performance of these main target attributes, the problem of tracking target drift is greatly alleviated. The OURS tracker has the best tracking performance on most attributes. In addition, we know from Figure 9 that the success rate of OURS tracker has obvious advantages compared with other trackers on most attributes. In summary, the OURS tracker achieves the best tracking performance compared with other trackers in almost all attributes.Figure 8
Detailed results on each attribute (OTB-100). Precision plots of (a) OPE-occlusion (48). (b) OPE-fast motion (39). (c) OPE-background clutter (31). (d) OPE-out-of-plane rotation (63). (e) OPE-in-plane rotation (51). (f) OPE-low resolution (9). (g) OPE-illumination variation (37). (h) OPE-occlusion (48). (i) OPE-motion blur (29). (j) OPE-out of view (14). (k) OPE-scale variation (63).
(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)Figure 9
Comparison of tracker success rates (OTB-100).


## 4.6. Experiments under Different Video Sequences

In order to more accurately describe the performance of the OURS tracker, this paper selects nine video sequences and tests their precision. The precision of nine video sequences in this experiment is shown in Table4, and the best results are reported in bold. According to the test results in Table 4, the OURS tracker has the highest precision when tracking Bolt, Coke, Couple, Jumping, Freeman1, Freeman4, Jogging, and Girl video sequences, and it also achieved near-highest precision when tracking Football video sequences. The OURS tracker performs better than other four trackers.Table 4
Performance (mean precision (20 px)) comparisons of different trackers in some video sequences.
TrackersBoltCokeCoupleJumpingFreeman1Freeman4JoggingGirlFootballOURS10096.789.783.595.975.597.410081.7Staple10091.834.944.937.112.536.610074.9KCF67.182.826.736.139.653.023.586.276.8CACF50.792.155.749.538.049.536.810078.8CSK23.591.210.620.140.838.954.253.683.4This experiment provides a qualitative comparison of OURS tracker with the existed tracker in order to verify the stability of OURS tracker. These video sequences constitute different challenging situations including illumination variation (IV), out-of-plane rotation (OPR), scale variation (SV), in-plane rotation (IPR), background clutter (BC), occlusion (OCC), deformation (DEF), fast motion (FM), and motion blur (MB). The actual tracking results of OURS tracker, CACF tracker, and KCF tracker in four video sequences are shown in Figure10. Figure 10(a) shows that the KCF tracker cannot stably track video sequences that contain scale variation attribute, but the OURS tracker and the CACF tracker can consistently track the Shaking video sequence. Figure 10(b) shows that the CACF tracker cannot successfully track the Bolt sequence when the video sequence contains in-plane rotation attributes, but the OURS tracker and the KCF tracker can handle these issues very well. Figure 10(c) shows that only the OURS tracker can stably track Freeman video sequences and neither the KCF tracker nor CACF tracker can successfully track the sequence. Figure 10(d) shows that only the OURS tracker can successfully track the Jumping video sequence when the video contains fast motion and motion blur properties. In summary, the OURS tracker has excellent stability.Figure 10
A visualization of the tracking results of OURS tracker and some visual trackers (CACF [25]and KCF [32]) on four benchmark sequences. (a) Shaking (IV, OPR, SV, and BC). (b) Bolt (OPR, OCC, DEF, and IPR). (c) Freeman1 (OPR, SV, and IPR). (d) Jumping (MB and FM).
(a)(b)(c)(d)


## 5. Conclusion

In this work, a scale-adaptive context-aware correlation filtering algorithm with constrained output response for object tracking is proposed: (1) this study assumes that the output response is Gaussian distribution, and a variable updating parameter is found according to Gaussian output constraints. (2) the filter is updated with variable updating parameters when the output response is Gaussian constraint, and the filter is updated with fixed updating parameters when the output response is not any Gaussian constraint. (3) the optimal scale of target is obtained by using the maximum posterior probability distribution.The proposed OURS tracker in this paper performs in the research cases because it has the following advantages. To begin with, the OURS tracker can provide a new filter that can get a more accurate model. In addition, the suggested tracker adopts a selectively updating strategy to effectively increase the tracking accuracy. Finally, the maximum posterior probability method is used to obtain a more accurate target scale. The experimental results have shown that the proposed tracker achieves better tracking performance than other trackers when dealing with drift problems caused by fast motion, deformation, and occlusion. Therefore, the developed tracker significantly improves the ability of the CACF tracker to handle drift problems and achieves better performance than many other trackers.Further research will focus on two aspects: (1) we will focus on solving the problem for low resolution and out-of-view to achieve higher tracking performance. (2) We are only studying the single target tracking problem, so the next step is to study the tracking of multiple targets with output response constraint which is satisfied with Gaussian distribution.



---

*Source: 4303725-2020-06-09.xml*