# Spatial Object Tracking Using an Enhanced Mean Shift Method Based on Perceptual Spatial-Space Generation Model

Authors: Pengcheng Han; Junping Du; Ming Fang
Journal: Journal of Applied Mathematics
 (2013)

---

## Abstract

Object tracking is one of the fundamental problems in computer vision, but existing efficient methods may not be suitable for spatial object tracking. Therefore, it is necessary to propose a more intelligent mathematical model. In this paper, we present an intelligent modeling method using an enhanced mean shift method based on a perceptual spatial-space generation model. We use a series of basic and composite graphic operators to complete signal perceptual transformation. The Monte Carlo contour detection method could overcome the dimensions problem of existing local filters. We also propose the enhanced mean shift method with estimation of spatial shape parameters. This method could adaptively adjust tracking areas and eliminate spatial background interference. Extensive experiments on a variety of spatial video sequences with comparison to several state-of-the-art methods demonstrate that our method could achieve reliable and accurate spatial object tracking.


---

## Body

## 1. Introduction

Mathematical formalism is probably the most precise and logical language in science research. It is typical for researchers in pure natural sciences to attempt to describe observed phenomena using mathematical correlations. However, because of real-world scenarios, it is often very difficult to construct a perfect and permanent mathematical model for one specific issue in engineering fields [1]. During the last years, the effort concentrated in self-optimizing and self-adaption was leading to a new field between mathematics and applications, called intelligent modeling [2]. In this paper, we propose a new intelligent modeling method using the enhanced mean shift method based on a perceptual spatial-space generation model for spatial object tracking. Object tracking has been applied to many fields, such as video surveillance [3], robot recognition [4], and traffic control [5]. In spatial on-orbit docking, object tracking could be used to track spatial aircraft and assist with ground control. Because the spatial images are mainly generated from low-rate videos [6] or airborne spectral imagery [7], which are captured by the aircraft sensors [8, 9], their resolution and spatial-temporal coverage are not very ideal. In addition, because of differences in the sensor spectral bands, acquisition position, and contrast gradient setting, there are shifts in the relative position and scale zoom in multisource images with the same scene. All this will bring influence to spatial object tracking results.Recently, multidimensional decomposition and multiscale representation methods have been widely applied to image processing and computer vision. Mumford and Gidas proposed the stochastic model [10], in which truncation errors and noise interference could be isolated from the discrete domain. Witkin and Koenderink proposed the image scale-space model [11, 12], which cleared noise interference in fine scales, and analysis errors could decrease in coarse scales. Burt and Lindeberg proposed the coarse-to-fine model [13, 14], which could reduce useless gradients in gradient entropy calculation. In the object tracking field, the traditional method is rectangular block region tagging [15, 16]. In [17], Isard and Blake applied a local filter to object tracking field. Sun and Liu proposed using a combination of the local description and global representation in object tracking [18]. Recently, a graphics model based on a Bayesian neural network was also applied to continuous object tracking [19]. However, if we applied these methods to spatial object tracking, spatial background clutter and moving object overlapping appeared in different scenes. The existing multiscale deviation will seriously reduce the spatial object tracking accuracy.In this paper, we propose a spatial object tracking method using an enhanced mean shift method based on a perceptual spatial-space generation model. We detect the spatial object continuity and saliency between different scales in the perceptual spatial-space generation model. The enhanced mean shift considers the relevance between motion area and static background. It could achieve more robust object tracking. Our proposed method is shown in Figure1. This paper is organized as follows. Section 2 describes the perceptual spatial-space generation model. Section 3 proposes an enhanced mean shift method. Section 4 shows experimental results. Section 5 is the conclusion.Figure 1
Enhanced mean shift method based on a perceptual spatial-space generation model.


## 2. Perceptual Spatial-Space Generation Model

Spatial images have a high amount of data structure, and their processing may not obey existing image processing model assumptions. In this paper, we propose a perceptual spatial-space generation model. It consists of two parts: a prototype pyramid, labeledS[0,n]=(S0,…,Sn), and a set of perceptual transform rules, labeled R[0,n-1]=(R0,R1,…,Rn-1). The set I[0,n] is the Gaussian transform pyramid. Our goal is to set a common variable p(I[0,n],S[0,n],R[0,n-1]) and a maximized posterior probability p(S[0,n],R[0,n-1]∣I[0,n]) that can be used to calculate the priority prototype pyramid and the perceptual transform rules.


### 2.1. Prototype Pyramid Generation

Generation model is a joint probability function with prototypeS and image I, and ΔR is a dictionary which includes image primitives, such as blobs, edges, crosses, and bars. It can be expressed as
(1)p(I;S;ΔR)=p(I∣S;ΔR)p(S).The decomposition probability could be divided into primitives and texture.Consider the following:(2)p(I∣S;ΔR)=∏k=1NRexp(-∑(u,v)∈R(I(u,v)-Rk(u,v)22σo2))·∏j=1NRp(IR,j∣IR;σj),whereS=〈V,E〉 is the properties graphics, V is the collection of primitives in S, (u,v) denoting the primitives in the dictionary Δ, and σ is the variance of the corresponding primitive features. The priority model p(S) is an uneven Gibbs model, which defines the graphical attributes on S. It focuses on continuity properties in the perceptual model, such as smoothness, continuity, and typical functions.Consider the following:(3)p(S)=exp{ε∑d=04∑(u,v)∈RRd|φ(u,v)|+(1-ε)∑(u,v)∈Rφ(Bu,Bv)},whereRd is the primitive mark in S and its connection degree is d. φ(Bu,Bv) is potential association for two correlation functions. Because there is uncertainty in the inner perception posterior probability, the prototype pyramid may not appear continuous for each layer calculation. In order to ensure transition and consistency of single frame diagram, we define a set of graphical operation factors
(4)Rk=(rk,1,rk,2,…,rk,m(k)),rk,i∈∑gram.Graphical operation factors could synthesize detected graphic edges into pairs of characteristic bridges. Each bridge is related with the properties of probability function. The conversion fromSk to Sk+1 is realized by a series of conversion rules Rk, and rule order could directly determine conversion efficiency. The generative rule graphic path from Sk to Sk+1could be expressed as
(5)p(Rk)=p(Sk+1∣Sk)=∏i=1m(k)[p(rk,i)+1mki∑j=1mki(wki,j-|dj-dki|)],p(I[0,n],S[0,n],R[0,n-1])=∏k=0np(Ik∣Sk;Δsk)·p(S0)·∏k=0n-1∏j=1m(k)p(rk,i),whereS is the optimal-calculated prototype. Under the condition that there will be no loss in perception model accuracy, we assume that Ssm begins to decay from S through the single operation factor. Ism will gradually reduce the resolution, and Ssm will also have a related complexity. The posterior probability could be expressed as
(6)P(sm)=logp(Ism∣Ssm)p(Ism∣S)+λsmlogp(Ssm)p(S)=logp(Ssm∣Ism)p(S∣Ism).A layered reduced perception model will not completely adapt to the complex modelS, so the first logarithmic ratio is often negative. The parameter λ is used to balance model fitness and complexity. If λ=1, we could launch simplified P(sm). λsm could be decided in the following range:
(7)0<logp(Ism∣Ssm)p(Ism∣S)+λsmlogp(Ssm)p(S)<1.The transform betweenSk and Sk+1is achieved using a group of greed detection. The accurate scale of graphical operation factor will make differences based on the subjective goal. We suppose that the graphical operation factor is between I and Ism(8)a1=-logp(Ism∣Ssm)p(Ism∣S),b1=λsmlogp(Ssm)p(S),a2=-logp(I∣Ssm)p(I∣S),b2=λsmlogp(Ssm)p(S).Based on formulas (6) and (7), we determined that the corresponding interval is
(9)N(|ra-rki,j|;0,σa2)<λsm<N(|rb-rbi,j|;0,σb2).


### 2.2. Perceptual Transform of Prototype Pyramid

In this section, our goal is to determine the optimum conversion path and deduce the hidden graphics prototype. Our method is scanning the prototype pyramid from top to bottom based on each primitive learning decision rule. Our method can be divided into three steps.Step 1 (prototype pyramid independent calculation).
We apply a pyramid algorithm in the bottom of imageI and calculate the Gaussian pyramid S0. Because each prototype layer is calculated using a MAP estimation [20], there is a certain loss in the continuity of the prototype pyramid. The specific formula is as follows:
(10)(S[0,n],R[0,n-1])=argmax∏k=0np(Ik∣Sk;Δk)·p(S0)∏k=1n∏j=1m(k)p(λk,j).Step 2 (pattern matching from bottom to up).
We match the image prototype attribute fromSk to Sk+1 using an image registration algorithm from a previous study [21]. We use x, y, z, and e as a judgment function to process each node i and obtain the related image characteristics at each time. Specifically, the matching degree between the ith node at the kth scale and the jth node at the (k+1)th scale can be expressed as
(11)match(i,j)=1Zexp{-(x(i)-x(j))22σx2-(y(i)-y(j))22σy2-(z(i)-z(j))22σz2-(e(i)-e(j))22σe2},
whereσ is the variance of related features. For similarity matching between Sk and Sk+1, this formulation allows the empty variable prototype to appear in Sk. We multiply Sk by the related variance σ and obtain a homologous Sk+1 with subsidiary value. Pattern matching results will be used as the initial parameter for the Markov chain matching in the next step.
Consider the following:(12)P(Sk,Sk+1)={((x;z)∈Match[vi(k),vi(k+1)]¯)(x;y;z;e)⟵S2:(kmin′<k<kmin′)×((x;z)∈match[vi(k),vi(k+1)]¯)}.Step 3 (Markov chain matching).
Due to the uncertainty for the initial perception in the posterior probability model and the relative complexity for dynamic graphic structure mining, we use the reversibility of Markov chain matching to match a perceptual transform. The Markov chain includes 25 pairs of reversible jump. In each path of Markov chain matching, there is a reversible jump betweenX1 and X2 [22]. These reversible jumps are related to their corresponding grammars, and each pair of these rules is based on probability selection. We use this mechanism to optimize the perception conversion path, which could lead a cross-scale continuous perception prediction.
Consider the following:(13)(S[0,n],Markov[0,n-1])=arg/max∏k=0np(Ik∣Sk;ΔR)·∏in∏jmg(ri,j).


### 2.3. Object Contour Evolution

The problem of such matching strategy is that it does not contain any matching, which could split a long contour into short edges. We propose a new Monte Carlo contour detection method. This method mainly chooses the right standard in certain scale using spatial-space domain knowledge. Our proposed method uses the weight set{xi,t,wi,t} to estimate the posterior probability density p(xi,t,wi,t). According to the resampling theory [23], it is feasible to calculate the sample specimen Q(x,w) with appropriate weights from the normal density distribution Xti.Consider the following:(14)wti,n=p(zti∣xti,ztN(i),x0:t-1i,z1:t-1i,Z1:t-1N(i))p(zti,Z1:t-1N(i)∣z1:t-1i)·Q(x,w).For the sequence sample, the important probabilityQ(x,w) can be chosen using the following mode:
(15)Q(x,w)=q(xti∣x0:t-1i,z1:ti,Z1:tN(i))q(x0:t-1i∣x1:t-1i,z1:ti,Z1:t-1N(i)).The entire probability can be approximated with a simplified formulation(16)wti,n=p(xti∣zti)p(xti,n∣zt-1i,n)q(xti∣x0:t-1i,z1:ti,Z1:tN(i))×∏j∈N(i){∑t=1Np(zti∣xtj,n)p(xtj,n∣ztj,n)}.We use only one part of the whole sample;p(xti,n∣zt-1i,n) simulates the interaction value between the two adjacent areas xti,n and xtj,l. The local probability p(ztj∣xtj,l) is the weight of the interactive area. In this paper, we use a sequence of Monte Carlo simulations to estimate the interactive part p(xtj∣xti) with its related estimation weight. The density importance function can be defined as follows:
(17)q(x0:ti∣xti,z1:ti,Z1:tN(i),Z1:tR(I))=q(xti∣x0:t-1i,z1:ti,Z1:tN(i),Z1:tR(I))×q(x0:t-1i∣x1:ti,z1:ti,Z1:t-1N(i),Z1:t-1R(I)).Based on the resampling theory, the sampling weight can be updated as follows:(18)wti,n=wt-1i,np(xti∣zti)p(xti,n∣zt-1i,n)q(xti∣x0:t-1i,z1:ti,Z1:tN(i),Z1:tR(I))×∏j∈N(i){∑t=1Np(zti∣xtj,n)p(xtj,n∣ztj,n)}×∏j∈N(i){∑L=1NLp(ztL∣xtk,L)p(xtk,L∣ztL,n)},whereN is the sample retrieval in the ith part, L is the sample length of the jth part, and ztL is the sample set of the jth part. As we use the Markov property, the density function p(ztk∣xtk,L) could do further approximation relied on the unit product of local observation similarity unit K.Consider the following:(19)p(ztk∣xtk,L)=p(ztN(k),ztR(L)∣xtk,x0:t-1L)=p(ztk∣xtk)·p(xtk,ZtN(k),ZtR(L)∣x0:t-1k)p(ztk,ZtN(k)∣Z1:t-1k,Z1:t-1N(t),Z1:t-1R(L)).In our proposed framework, all parts of the detected object will be tracked at the same time. There is no need to calculate all unit probabilitiesp(ztk∣xtk,L), so we can use the local possibility and directly estimate the weight of all units.


### 2.4. Markov Random Field Representation

We propose Markov random field representation for perception generation model. We define the pixel perception setX, in which any two arbitrary pixels are adjacent. The adjacent relation is an interactive relationship; in the an adjacent pixels system E={δ(x,y):(x,y)∈I}, if (x,y) is adjacent point of (w,v), then (w,v) is also adjacent with (x,y), and p(I) is the Markov random field with respect to the perception set X(20)p(Ix,y∣X(x,y))=p(Ix,y∣I(δ∖X(x,y))).We can defineI(δ) as the pixels set that contains all pixels of X. The Markov property mentioned in formula (20) relies on the density distribution of the adjacent pixels. According to S. Geman and D. Geman [24], the Markov random field-related pixels in system X can be rewritten as the following Gibbs distribution:
(21)p(I)=12Z|ΣC|1/2exp{-∑XGx(I(E))·∑C-1(Exi-Eyi)},whereGx(I(E)) is the potential variable function defined in I(δ). Z is a conventional constant, which can maintain the p(I) sum up to 1. For spatial object tracking, the perception latent variables that appear in pairs are difficult to describe accurately. If X has an amount of pixels, then Gx will be a multidimensional function. Here, we use the topological model to solve the above problems. Assuming that the spatial perceptual rules set is φ(I), then φx,y(I) could extract the local characteristics near pixels (x,y). A specific example is φx,y(I)=〈I,βx,y,s〉, s=(I,θ). We transform the image I with the Gibbs distribution
(22)GD(I∣β)=1Zexp{GD(p(I(k)))-GD′(p(X))}=1Zexp{∑s=1S∑x,yβx,y,s(Xx,y,s(I))-∑x,yp(I)px,y,s·(I-p(I))},whereβx,y,s is the low-dimensional image characteristic function set, Z is the conventional constant that depends on β, and Xx,y,s=(I,βx,y,s). Assuming that the normal distribution is p(I), px,y,s is the normal distribution of βx,y,s under I-p(I). β could guarantee f*(I)=f(I|β). For each (x,y,s), the marginal distribution of φ(x,y,s) is p(x,y,s). With any given f=f(I|β),D(p|f*)≤D(p|f), the specific calculation of the axiom is as follows:
(23)D(p∣f)-D(p∣f*)=logZ(β)Z(β*){∑s=1S∑x,yβx,y,s*(Xx,y,k(I))-βx,y,s(Xx,y,k(I))}=D(f*∣∣f)≥0.In the formulation off(I|β),f* can be seen as the best approximation of probability p, and it also can be marked as the “maximum likelihood.” From the minimized D(p||f(I|β)) to the maximized E[logf(I|β)], the marginal probability φ(x,y,s) can be px,y,s() for any (x,y,s) under p(I). So H(f*)-H(f)=D(p||f*).Consider the following:(24)H(f*)-H(f)=Eq[logq(I)]-Ef*[logf*(I)]=Eq[logq(I)]-Eq[logf*(I)]=D(q||f*).In order to model the observed image, we assume thatβx,y,s()=βs(), β does not depend on (x,y). We can continue to parameterize the βs or normalize β in a low-dimensional scale. If we normalize βs in S=1,…,L and make βs(φx,y,s(I))=βsl, we could rewrite the Gibbs distribution as follows:
(25)GD(I∣β)=1Zexp{∑s=1S∑x,y∑lβx,y,s(Xx,y,s(I))-∑x,y∑lp(I)p(x,y,s)·(I-p(I))}=1Zexp{∑s∑lβslHkl(I)-∑x,y∑lp′(I)·Hk(I)}=1Zexp{∑s〈βk,Hk(I)〉},whereHk(I)=px,yφx,y,s(I) represents the quantity of the effective points φx,y,s(I) which falls into the interval S and Hk=(Hk,L) is the marginal matrix related to {φx,y,s,S}. If we want to find the maximum estimate coefficient β, we need to calculate the spatial-scale statistical coefficient Hk(I)(26)Eβ[Hk(I)]={1n∑k=1nH(p(I(k)))-Hk(Iobserved)}.In other words, we need to match the spatial data and the related model. The most suitable model is determined byHk(Iobserved), and Eβ[Hk(I)] is an average parameter; value β is a natural parameter. In the perceptual model, there will be a global balance variable. We can make a minor adjustment
(27)p(I0∣Iρ0)=1Zexp{∑k∑x,y∈p0βk(Φx,y,s)}+∑kEw[logp(w∣Is)p(w∣F)],whereIρ0 is the approximation value of global observation estimated from observed spatial images. The local equilibrium parameter defined in any pixel will obey the specific distribution
(28)p(I)=1Zexp{∑k=1K(Cm,kBk+εm-ηjFj(I)Bk)+λ∑k=1KS(Cm,k)},whereηj is local approximation for pixel p0. It is the only distribution that has similar effects with perceptual model.


## 2.1. Prototype Pyramid Generation

Generation model is a joint probability function with prototypeS and image I, and ΔR is a dictionary which includes image primitives, such as blobs, edges, crosses, and bars. It can be expressed as
(1)p(I;S;ΔR)=p(I∣S;ΔR)p(S).The decomposition probability could be divided into primitives and texture.Consider the following:(2)p(I∣S;ΔR)=∏k=1NRexp(-∑(u,v)∈R(I(u,v)-Rk(u,v)22σo2))·∏j=1NRp(IR,j∣IR;σj),whereS=〈V,E〉 is the properties graphics, V is the collection of primitives in S, (u,v) denoting the primitives in the dictionary Δ, and σ is the variance of the corresponding primitive features. The priority model p(S) is an uneven Gibbs model, which defines the graphical attributes on S. It focuses on continuity properties in the perceptual model, such as smoothness, continuity, and typical functions.Consider the following:(3)p(S)=exp{ε∑d=04∑(u,v)∈RRd|φ(u,v)|+(1-ε)∑(u,v)∈Rφ(Bu,Bv)},whereRd is the primitive mark in S and its connection degree is d. φ(Bu,Bv) is potential association for two correlation functions. Because there is uncertainty in the inner perception posterior probability, the prototype pyramid may not appear continuous for each layer calculation. In order to ensure transition and consistency of single frame diagram, we define a set of graphical operation factors
(4)Rk=(rk,1,rk,2,…,rk,m(k)),rk,i∈∑gram.Graphical operation factors could synthesize detected graphic edges into pairs of characteristic bridges. Each bridge is related with the properties of probability function. The conversion fromSk to Sk+1 is realized by a series of conversion rules Rk, and rule order could directly determine conversion efficiency. The generative rule graphic path from Sk to Sk+1could be expressed as
(5)p(Rk)=p(Sk+1∣Sk)=∏i=1m(k)[p(rk,i)+1mki∑j=1mki(wki,j-|dj-dki|)],p(I[0,n],S[0,n],R[0,n-1])=∏k=0np(Ik∣Sk;Δsk)·p(S0)·∏k=0n-1∏j=1m(k)p(rk,i),whereS is the optimal-calculated prototype. Under the condition that there will be no loss in perception model accuracy, we assume that Ssm begins to decay from S through the single operation factor. Ism will gradually reduce the resolution, and Ssm will also have a related complexity. The posterior probability could be expressed as
(6)P(sm)=logp(Ism∣Ssm)p(Ism∣S)+λsmlogp(Ssm)p(S)=logp(Ssm∣Ism)p(S∣Ism).A layered reduced perception model will not completely adapt to the complex modelS, so the first logarithmic ratio is often negative. The parameter λ is used to balance model fitness and complexity. If λ=1, we could launch simplified P(sm). λsm could be decided in the following range:
(7)0<logp(Ism∣Ssm)p(Ism∣S)+λsmlogp(Ssm)p(S)<1.The transform betweenSk and Sk+1is achieved using a group of greed detection. The accurate scale of graphical operation factor will make differences based on the subjective goal. We suppose that the graphical operation factor is between I and Ism(8)a1=-logp(Ism∣Ssm)p(Ism∣S),b1=λsmlogp(Ssm)p(S),a2=-logp(I∣Ssm)p(I∣S),b2=λsmlogp(Ssm)p(S).Based on formulas (6) and (7), we determined that the corresponding interval is
(9)N(|ra-rki,j|;0,σa2)<λsm<N(|rb-rbi,j|;0,σb2).


## 2.2. Perceptual Transform of Prototype Pyramid

In this section, our goal is to determine the optimum conversion path and deduce the hidden graphics prototype. Our method is scanning the prototype pyramid from top to bottom based on each primitive learning decision rule. Our method can be divided into three steps.Step 1 (prototype pyramid independent calculation).
We apply a pyramid algorithm in the bottom of imageI and calculate the Gaussian pyramid S0. Because each prototype layer is calculated using a MAP estimation [20], there is a certain loss in the continuity of the prototype pyramid. The specific formula is as follows:
(10)(S[0,n],R[0,n-1])=argmax∏k=0np(Ik∣Sk;Δk)·p(S0)∏k=1n∏j=1m(k)p(λk,j).Step 2 (pattern matching from bottom to up).
We match the image prototype attribute fromSk to Sk+1 using an image registration algorithm from a previous study [21]. We use x, y, z, and e as a judgment function to process each node i and obtain the related image characteristics at each time. Specifically, the matching degree between the ith node at the kth scale and the jth node at the (k+1)th scale can be expressed as
(11)match(i,j)=1Zexp{-(x(i)-x(j))22σx2-(y(i)-y(j))22σy2-(z(i)-z(j))22σz2-(e(i)-e(j))22σe2},
whereσ is the variance of related features. For similarity matching between Sk and Sk+1, this formulation allows the empty variable prototype to appear in Sk. We multiply Sk by the related variance σ and obtain a homologous Sk+1 with subsidiary value. Pattern matching results will be used as the initial parameter for the Markov chain matching in the next step.
Consider the following:(12)P(Sk,Sk+1)={((x;z)∈Match[vi(k),vi(k+1)]¯)(x;y;z;e)⟵S2:(kmin′<k<kmin′)×((x;z)∈match[vi(k),vi(k+1)]¯)}.Step 3 (Markov chain matching).
Due to the uncertainty for the initial perception in the posterior probability model and the relative complexity for dynamic graphic structure mining, we use the reversibility of Markov chain matching to match a perceptual transform. The Markov chain includes 25 pairs of reversible jump. In each path of Markov chain matching, there is a reversible jump betweenX1 and X2 [22]. These reversible jumps are related to their corresponding grammars, and each pair of these rules is based on probability selection. We use this mechanism to optimize the perception conversion path, which could lead a cross-scale continuous perception prediction.
Consider the following:(13)(S[0,n],Markov[0,n-1])=arg/max∏k=0np(Ik∣Sk;ΔR)·∏in∏jmg(ri,j).


## 2.3. Object Contour Evolution

The problem of such matching strategy is that it does not contain any matching, which could split a long contour into short edges. We propose a new Monte Carlo contour detection method. This method mainly chooses the right standard in certain scale using spatial-space domain knowledge. Our proposed method uses the weight set{xi,t,wi,t} to estimate the posterior probability density p(xi,t,wi,t). According to the resampling theory [23], it is feasible to calculate the sample specimen Q(x,w) with appropriate weights from the normal density distribution Xti.Consider the following:(14)wti,n=p(zti∣xti,ztN(i),x0:t-1i,z1:t-1i,Z1:t-1N(i))p(zti,Z1:t-1N(i)∣z1:t-1i)·Q(x,w).For the sequence sample, the important probabilityQ(x,w) can be chosen using the following mode:
(15)Q(x,w)=q(xti∣x0:t-1i,z1:ti,Z1:tN(i))q(x0:t-1i∣x1:t-1i,z1:ti,Z1:t-1N(i)).The entire probability can be approximated with a simplified formulation(16)wti,n=p(xti∣zti)p(xti,n∣zt-1i,n)q(xti∣x0:t-1i,z1:ti,Z1:tN(i))×∏j∈N(i){∑t=1Np(zti∣xtj,n)p(xtj,n∣ztj,n)}.We use only one part of the whole sample;p(xti,n∣zt-1i,n) simulates the interaction value between the two adjacent areas xti,n and xtj,l. The local probability p(ztj∣xtj,l) is the weight of the interactive area. In this paper, we use a sequence of Monte Carlo simulations to estimate the interactive part p(xtj∣xti) with its related estimation weight. The density importance function can be defined as follows:
(17)q(x0:ti∣xti,z1:ti,Z1:tN(i),Z1:tR(I))=q(xti∣x0:t-1i,z1:ti,Z1:tN(i),Z1:tR(I))×q(x0:t-1i∣x1:ti,z1:ti,Z1:t-1N(i),Z1:t-1R(I)).Based on the resampling theory, the sampling weight can be updated as follows:(18)wti,n=wt-1i,np(xti∣zti)p(xti,n∣zt-1i,n)q(xti∣x0:t-1i,z1:ti,Z1:tN(i),Z1:tR(I))×∏j∈N(i){∑t=1Np(zti∣xtj,n)p(xtj,n∣ztj,n)}×∏j∈N(i){∑L=1NLp(ztL∣xtk,L)p(xtk,L∣ztL,n)},whereN is the sample retrieval in the ith part, L is the sample length of the jth part, and ztL is the sample set of the jth part. As we use the Markov property, the density function p(ztk∣xtk,L) could do further approximation relied on the unit product of local observation similarity unit K.Consider the following:(19)p(ztk∣xtk,L)=p(ztN(k),ztR(L)∣xtk,x0:t-1L)=p(ztk∣xtk)·p(xtk,ZtN(k),ZtR(L)∣x0:t-1k)p(ztk,ZtN(k)∣Z1:t-1k,Z1:t-1N(t),Z1:t-1R(L)).In our proposed framework, all parts of the detected object will be tracked at the same time. There is no need to calculate all unit probabilitiesp(ztk∣xtk,L), so we can use the local possibility and directly estimate the weight of all units.


## 2.4. Markov Random Field Representation

We propose Markov random field representation for perception generation model. We define the pixel perception setX, in which any two arbitrary pixels are adjacent. The adjacent relation is an interactive relationship; in the an adjacent pixels system E={δ(x,y):(x,y)∈I}, if (x,y) is adjacent point of (w,v), then (w,v) is also adjacent with (x,y), and p(I) is the Markov random field with respect to the perception set X(20)p(Ix,y∣X(x,y))=p(Ix,y∣I(δ∖X(x,y))).We can defineI(δ) as the pixels set that contains all pixels of X. The Markov property mentioned in formula (20) relies on the density distribution of the adjacent pixels. According to S. Geman and D. Geman [24], the Markov random field-related pixels in system X can be rewritten as the following Gibbs distribution:
(21)p(I)=12Z|ΣC|1/2exp{-∑XGx(I(E))·∑C-1(Exi-Eyi)},whereGx(I(E)) is the potential variable function defined in I(δ). Z is a conventional constant, which can maintain the p(I) sum up to 1. For spatial object tracking, the perception latent variables that appear in pairs are difficult to describe accurately. If X has an amount of pixels, then Gx will be a multidimensional function. Here, we use the topological model to solve the above problems. Assuming that the spatial perceptual rules set is φ(I), then φx,y(I) could extract the local characteristics near pixels (x,y). A specific example is φx,y(I)=〈I,βx,y,s〉, s=(I,θ). We transform the image I with the Gibbs distribution
(22)GD(I∣β)=1Zexp{GD(p(I(k)))-GD′(p(X))}=1Zexp{∑s=1S∑x,yβx,y,s(Xx,y,s(I))-∑x,yp(I)px,y,s·(I-p(I))},whereβx,y,s is the low-dimensional image characteristic function set, Z is the conventional constant that depends on β, and Xx,y,s=(I,βx,y,s). Assuming that the normal distribution is p(I), px,y,s is the normal distribution of βx,y,s under I-p(I). β could guarantee f*(I)=f(I|β). For each (x,y,s), the marginal distribution of φ(x,y,s) is p(x,y,s). With any given f=f(I|β),D(p|f*)≤D(p|f), the specific calculation of the axiom is as follows:
(23)D(p∣f)-D(p∣f*)=logZ(β)Z(β*){∑s=1S∑x,yβx,y,s*(Xx,y,k(I))-βx,y,s(Xx,y,k(I))}=D(f*∣∣f)≥0.In the formulation off(I|β),f* can be seen as the best approximation of probability p, and it also can be marked as the “maximum likelihood.” From the minimized D(p||f(I|β)) to the maximized E[logf(I|β)], the marginal probability φ(x,y,s) can be px,y,s() for any (x,y,s) under p(I). So H(f*)-H(f)=D(p||f*).Consider the following:(24)H(f*)-H(f)=Eq[logq(I)]-Ef*[logf*(I)]=Eq[logq(I)]-Eq[logf*(I)]=D(q||f*).In order to model the observed image, we assume thatβx,y,s()=βs(), β does not depend on (x,y). We can continue to parameterize the βs or normalize β in a low-dimensional scale. If we normalize βs in S=1,…,L and make βs(φx,y,s(I))=βsl, we could rewrite the Gibbs distribution as follows:
(25)GD(I∣β)=1Zexp{∑s=1S∑x,y∑lβx,y,s(Xx,y,s(I))-∑x,y∑lp(I)p(x,y,s)·(I-p(I))}=1Zexp{∑s∑lβslHkl(I)-∑x,y∑lp′(I)·Hk(I)}=1Zexp{∑s〈βk,Hk(I)〉},whereHk(I)=px,yφx,y,s(I) represents the quantity of the effective points φx,y,s(I) which falls into the interval S and Hk=(Hk,L) is the marginal matrix related to {φx,y,s,S}. If we want to find the maximum estimate coefficient β, we need to calculate the spatial-scale statistical coefficient Hk(I)(26)Eβ[Hk(I)]={1n∑k=1nH(p(I(k)))-Hk(Iobserved)}.In other words, we need to match the spatial data and the related model. The most suitable model is determined byHk(Iobserved), and Eβ[Hk(I)] is an average parameter; value β is a natural parameter. In the perceptual model, there will be a global balance variable. We can make a minor adjustment
(27)p(I0∣Iρ0)=1Zexp{∑k∑x,y∈p0βk(Φx,y,s)}+∑kEw[logp(w∣Is)p(w∣F)],whereIρ0 is the approximation value of global observation estimated from observed spatial images. The local equilibrium parameter defined in any pixel will obey the specific distribution
(28)p(I)=1Zexp{∑k=1K(Cm,kBk+εm-ηjFj(I)Bk)+λ∑k=1KS(Cm,k)},whereηj is local approximation for pixel p0. It is the only distribution that has similar effects with perceptual model.


## 3. Enhanced Mean Shift Method

### 3.1. Mean Shift

In this section, we propose an enhanced mean shift method. Our method uses the mathematical recursion method [25]. The spatial tracking object will be represented by a spatial histogram consisting of a weighted evaluation, in which the probability estimation function p(y) and q(x0) are used to represent the potential motion probability in images I(y) and I(x0). The histogram variables can be expressed as follows:
(29)p(y)=c|Σ|2∑ik(yi∑yiT)δ[bu(I(yi))-u]+c2|Σ|1/2∑wj,k(yjT∑-1yj),q(x0)=c|Σ|2∑jk(xj∑xjT)δ[bu(I(xj))-u]+c2|Σ|1/2∑wj,k(yjT∑-1yj),whereyi=(yTi-y), xj=(xTj-x), and Σ are the representatives of weight function, bu(I(yi)) and bu(I(yj)) are motion estimations for positions yi and yj, wj is the box parameter, C is normalized constant, u=1,…,m, m contains all the binary pixels, and y and x are the center of related kernel function. The Bhattacharyya coefficient ρ will be used to detect similarity between tracking object area and the potential background.Consider the following:(30)ρ(p,q)=∑upy(x,y,Σ)qx∥Lx(i)∥×∥Ly(j)∥min(∥Ly(j)∥∥Lx(i)∥,∥Lx(i)∥∥Ly(j)∥).We apply the first-order Taylor sequence extension, in which(x,y) are the coordinates of the center position in the previous frame, and then we obtain the following extended formulas:
(31)ρ=∑u12pyqx(x,y,Σ)+c2|Σ|2∑jωjk(yjT∑xiT),ωj=∑upyqxδ[bu(I(yj))-u].The center of kernel function can be determined by the estimate ofρ(x,y)(32)δ=α1∑i=1nρ(xpi,xi)+α2∑i=2n|ρ(xpi,xpl)-ρ(xi,xl)|.In order to estimate the kernel function, the normalized bandwidth will be applied to a similarity judgment. The normalized bandwidth can be obtained by estimating|Σ|ρ(x,y)(33)∑ρ(x,y)=21-r∑j=1nωjxj(yjT∑-1xiT)g∑j=1nωjk(yjT∑-1xiT)+1Npre∑i=1Npremaxj〈Dprei,Dcurj〉,whereyjT=(yj-y) could accurately determine ρ(x,y). Equations (32) and (33) will be calculated in an alternative iteration until the estimated parameters can cover all the variables.


### 3.2. Estimation of Spatial Shape Parameters

We also use the iterated function to determine boundary parameterVT(2), which contains five fully adjustable affine box parameters. These parameters are the width, height, length, orientation, and center location. The orientation θ will be defined as the angle between the horizontal and width matrix. The box with width w and height h is the ellipse area between the long and short coordinate system. The relationship between θ, h, w and the bandwidth matrix Σ can be expressed as follows:
(34)Σ=RT(θ)[(h2)200(ω2)2]R(θ),whereR=[cosθ-sinθsinθcosθ].These parameters can be calculated using the octave decomposition method [26]; the specific formula is as follows:(35)P(ftpre∣ftcur)={N(ftpre:ftcur,diag[σw2,σh2,σl2,σo2,σc2]),ifftcurisnotnull,Pv1,ifftcurisnull,whereftpre and ftcurare the octave decomposition components of the previous frame and the current frame.


### 3.3. Spatial Object Tracking Using Enhanced Mean Shift

In order to limit the possibility that background pixels appear in the tracking object, we use a relatively small elliptical area, in which the contract domain is determined by the factorK; in our experiments, K=0.7. The elliptical area can be defined as follows:
(36)EA=1-∑i=1M∑upui,(j)(y,Σ)qui,(j)+12∑i=1MCM|Σi|1/2∑j=1Nwjvk(yjT∑v-1yj).In our proposed method, we should determine whether the previous frame motion area (ObjectA) will be used to guide the next frame object (Object B) tracking. If the number of continuous characteristic pixels and the Bhattacharyya coefficient for A are both higher, the initial rectangular tracking area for B will refer to A’s settings, and the tracking area will be determined by the previous frame mean shift.Consider the following:(37)Vt(2)={Vt(1),ifdistt=∑k=14∥xt,k-xt-1,k∥2<T1(2),Vt-1(2),ifdistt=∑k=14∥xt+1,k-xt,k∥2<T2(2),whereT1(2) and T2(2)determine the threshold value. In order to solve drift and error propagation, we use enhanced mean shift for frame resampling. The object tracking resampling operation can be summarized as follows:
(38)ρ≈∑i=1M∑u12puiqui∥xt,i(2)-xt-1,i(2)∥+∑i=1M∑yt∈RtRt-1(obj)2|Σi|1/2wtiVt-1(obj)(yjT∑i-1yj),whereRt⟵Rt-1(obj),Vt⟵Vt-1(obj),ρt⟵0,wherext,I(2) and xt-1,I(2) are the four-dimensional motion areas in frames t and t-1 and T3(2)and T4(2) are the threshold values determined by the graphic distance and the similarity shape.


## 3.1. Mean Shift

In this section, we propose an enhanced mean shift method. Our method uses the mathematical recursion method [25]. The spatial tracking object will be represented by a spatial histogram consisting of a weighted evaluation, in which the probability estimation function p(y) and q(x0) are used to represent the potential motion probability in images I(y) and I(x0). The histogram variables can be expressed as follows:
(29)p(y)=c|Σ|2∑ik(yi∑yiT)δ[bu(I(yi))-u]+c2|Σ|1/2∑wj,k(yjT∑-1yj),q(x0)=c|Σ|2∑jk(xj∑xjT)δ[bu(I(xj))-u]+c2|Σ|1/2∑wj,k(yjT∑-1yj),whereyi=(yTi-y), xj=(xTj-x), and Σ are the representatives of weight function, bu(I(yi)) and bu(I(yj)) are motion estimations for positions yi and yj, wj is the box parameter, C is normalized constant, u=1,…,m, m contains all the binary pixels, and y and x are the center of related kernel function. The Bhattacharyya coefficient ρ will be used to detect similarity between tracking object area and the potential background.Consider the following:(30)ρ(p,q)=∑upy(x,y,Σ)qx∥Lx(i)∥×∥Ly(j)∥min(∥Ly(j)∥∥Lx(i)∥,∥Lx(i)∥∥Ly(j)∥).We apply the first-order Taylor sequence extension, in which(x,y) are the coordinates of the center position in the previous frame, and then we obtain the following extended formulas:
(31)ρ=∑u12pyqx(x,y,Σ)+c2|Σ|2∑jωjk(yjT∑xiT),ωj=∑upyqxδ[bu(I(yj))-u].The center of kernel function can be determined by the estimate ofρ(x,y)(32)δ=α1∑i=1nρ(xpi,xi)+α2∑i=2n|ρ(xpi,xpl)-ρ(xi,xl)|.In order to estimate the kernel function, the normalized bandwidth will be applied to a similarity judgment. The normalized bandwidth can be obtained by estimating|Σ|ρ(x,y)(33)∑ρ(x,y)=21-r∑j=1nωjxj(yjT∑-1xiT)g∑j=1nωjk(yjT∑-1xiT)+1Npre∑i=1Npremaxj〈Dprei,Dcurj〉,whereyjT=(yj-y) could accurately determine ρ(x,y). Equations (32) and (33) will be calculated in an alternative iteration until the estimated parameters can cover all the variables.


## 3.2. Estimation of Spatial Shape Parameters

We also use the iterated function to determine boundary parameterVT(2), which contains five fully adjustable affine box parameters. These parameters are the width, height, length, orientation, and center location. The orientation θ will be defined as the angle between the horizontal and width matrix. The box with width w and height h is the ellipse area between the long and short coordinate system. The relationship between θ, h, w and the bandwidth matrix Σ can be expressed as follows:
(34)Σ=RT(θ)[(h2)200(ω2)2]R(θ),whereR=[cosθ-sinθsinθcosθ].These parameters can be calculated using the octave decomposition method [26]; the specific formula is as follows:(35)P(ftpre∣ftcur)={N(ftpre:ftcur,diag[σw2,σh2,σl2,σo2,σc2]),ifftcurisnotnull,Pv1,ifftcurisnull,whereftpre and ftcurare the octave decomposition components of the previous frame and the current frame.


## 3.3. Spatial Object Tracking Using Enhanced Mean Shift

In order to limit the possibility that background pixels appear in the tracking object, we use a relatively small elliptical area, in which the contract domain is determined by the factorK; in our experiments, K=0.7. The elliptical area can be defined as follows:
(36)EA=1-∑i=1M∑upui,(j)(y,Σ)qui,(j)+12∑i=1MCM|Σi|1/2∑j=1Nwjvk(yjT∑v-1yj).In our proposed method, we should determine whether the previous frame motion area (ObjectA) will be used to guide the next frame object (Object B) tracking. If the number of continuous characteristic pixels and the Bhattacharyya coefficient for A are both higher, the initial rectangular tracking area for B will refer to A’s settings, and the tracking area will be determined by the previous frame mean shift.Consider the following:(37)Vt(2)={Vt(1),ifdistt=∑k=14∥xt,k-xt-1,k∥2<T1(2),Vt-1(2),ifdistt=∑k=14∥xt+1,k-xt,k∥2<T2(2),whereT1(2) and T2(2)determine the threshold value. In order to solve drift and error propagation, we use enhanced mean shift for frame resampling. The object tracking resampling operation can be summarized as follows:
(38)ρ≈∑i=1M∑u12puiqui∥xt,i(2)-xt-1,i(2)∥+∑i=1M∑yt∈RtRt-1(obj)2|Σi|1/2wtiVt-1(obj)(yjT∑i-1yj),whereRt⟵Rt-1(obj),Vt⟵Vt-1(obj),ρt⟵0,wherext,I(2) and xt-1,I(2) are the four-dimensional motion areas in frames t and t-1 and T3(2)and T4(2) are the threshold values determined by the graphic distance and the similarity shape.


## 4. Experimental Result

We conducted experiments on four different spatial video sequences, in which tracking objects are spatial satellites and aircrafts. We uniformed the sequence image size for the same spatial resolution; each frame is320×256. The superiority of our proposed algorithm will be validated by an intuitive performance and objective evaluation.


### 4.1. Contour Evolution in Prototype Pyramid

Contour evolution experimental results are shown in Figure2. Each prototype pyramid layer is calculated independently. The performances have shown that the contour evolution has a better continuity in the layer-by-layer prototype pyramid, and the approximate effects are derived from a perceptual spatial-space generation model and are closer to the human visual perception system.Contour evolution on spatial video sequences. (a) Original video frame and (b)–(d) contour evolution from a coarse to fine scale.
(a)
(b)
(c)
(d)


### 4.2. Markov Random Field Representation

The Markov random field representation is shown in Figure3. The significant representative region derivate from the perceptual spatial-space generation model contains significant feature information, which is closely related to the different color and texture distribution in the spatial area. The experimental results show that the region representation effect, which has been smoothed, could highlight the sensitivity of the motion area more than the initial representation.Markov random field representation in the perceptual spatial-space generation model. (a) Original frame and (b)–(c) initial and global smoothing representation.
(a)
(b)
(c)


### 4.3. Enhanced Mean Shift Object Tracking

We conducted experiments on the ten video sequences, in which the tracking objects include spatial satellite and aircraft, highway and park surveillance, and the human body. The enhanced mean shift conducted an iterative calculation 20 times on each video sequence. The normalized matrix bandwidth parameters are determined by a different experimental sample. In our experiments, it is 0.63 for video sequences 1, 2, and 3, 0.42 for video sequences 4 and 6, 0.56 for video sequences 5 and 8, 0.21 for video sequence 7,  and 0.60 for video sequence 9 and 10.


#### 4.3.1. Satellite-1, 2, and 3

Tracker-1: the particle filter will have a negative impact on the horizontal direction, and the motion estimation will appear noncontinuous. Tracker-2: the distance metric learning will affect motion area determination, and the rectangular window tracking will produce some deviation. Our proposed algorithm can better track the spatial objects, and satellites and aircraft can be completely contained in the rectangular window with a similar color distribution and background quiver. The edge deviation variance can be controlled well using spatial object detection, with no offset and blurring.


#### 4.3.2. Automobile-1 and 2

In a nighttime environment, as the weak light and brightness, Tracker-1 and Tracker-2 obtain a vague tracking result, and the confusion area between background and the tracking object becomes larger. In the Automobile-2 sequence in particular, as the illumination from other automotive foreground lamps, Track-1 produces particularly serious deviation. Our proposed enhanced mean shift method could distinguish the tracking object from background confusion, and the tracking results do not appear to obviously deviate.


#### 4.3.3. Highway

Tracker-1: this method will lose the tracking center in some frames. There are some cross-rectangular windows between the far and close vision sequences. Shape errors exist in the rectangle window estimations. Tracker-2: the tracking results also have a tracking deviation in the far vision. Our proposed method can maintain consistency between different visions and does not appear to have huge deviations.


#### 4.3.4. Running, Walking

There are no obvious differences between the different methods. Except for some slow motion (walking body), Tracker-2 shows some partial deviations.


#### 4.3.5. Automobile-3, Walking-2

These cases are used to further test the robustness of our proposed method in the scene containing two or more moving objects. We could observe that Tracker-2 results in less accurate boxes probably due to its poor estimate of different moving object center position in the same scene. The performance of Tracker-1 is somewhat better; however, it also produces partial deviations. Our proposed method is more robust when dealing with scenarios of two or more moving objects.


### 4.4. Objective Evaluation

We use three objective evaluations for evaluating the object tracking performance (Figure4).Figure 4
Experimental results from the tracking, marked by the bounding box. The red (solid line) box is from our proposed method, the blue (dashed line) box is Tracker-1 from the particle filters in [27], and the green (dashed line) box is Tracker-2 from the online distance metric learning in [28]. The rows 1–10 contain images from the videos “Satellite-1,” “Satellite-2,” “Satellite-3,” “Automobile-1,” “Running,” “Automobile-2,” “Highway,” “Walking,” “Automobile-3” and “Walking-2,” respectively.


#### 4.4.1. Euclidian Distance

The Euclidian distance is the distance between rectangular windows obtained by tracking methods and the artificially marked. The specific calculation is as follows:(39)D=14∑i=14(xA,i-xGT,i)2+(yA,i-yGT,i)2,where(xA,i,yA,i), (xGT,i,yGT,i), and  i=1,2,3,4 are corner coordinates of the rectangular window calculated by tracking methods and artificially marked. Figure 5 shows the Euclidian distance between the tracked and artificially marked areas for our proposed method and the two other trackers on the videos.Figure 5
Euclidian distances between the tracked and artificially marked areas. Red curve: our proposed method; blue curve:Tracker-1 in [27]; green curve: Tracker-2 in [28].The averaged Euclidian distance calculated on all videos is shown in Table1. Compared to the distance values from Tracker-1 and Tracker-2, our proposed method clearly shows smaller and bounded Euclidian distances for the tested videos.Table 1
Results of the averaged Euclidian distance over all frames in each video.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
38.41
39.60
17.82
Satellite-2
60.97
60.44
23.45
Satellite-3
65.41
56.37
23.32
Automobile-1
27.64
28.22
14.23
Running
24.53
25.76
16.44
Automobile-2
52.67
59.21
32.87
Highway
22.16
21.06
10.54
Walking
25.29
23.19
12.73
Automobile-3
36.53
36.47
25.71
Walking-2
33.46
33.43
23.42


#### 4.4.2. Mean Square Errors (MSEs)

We have(40)MSE=1N∑i=1N(xiA-xiGT)2+(yiA-yiGT)2,where(xiA,yiA), (xiGT,yiGT) is the center of the tracking area obtained by methods and artificially marked, respectively. N is the total number of video sequence frames. The experimental results are shown in Table 2 and, it can be seen that our proposed method has the minimum MSE, which means it has the lowest tracking deviation. Our method has obvious advantages compared to other methods.Table 2
Results of averaged MSE errors for the tracked box from our proposed method,Tracker-1, and Tracker-2.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
2.7865
6.2853
1.5866
Satellite-2
5.2368
13.4572
3.2914
Satellite-3
15.7326
12.3574
5.1233
Automobile-1
6.2357
7.8211
2.5671
Running
8.4596
7.7324
1.9635
Automobile-2
14.3687
18.5964
4.2158
Highway
4.2365
3.2151
1.1623
Walking
7.5642
9.2534
3.1486
Automobile-3
9.7749
10.1037
4.3357
Walking-2
8.9681
9.6614
4.0654


#### 4.4.3. Bhattacharyya Distance

The Bhattacharyya distance is used to judge the deviation degree between the tracking area and the actual motion area. The specific calculation method is as follows:meanA and meanGT are the mean vectors with respect to the tracking area and are calculated by our method and artificially marked. The variables covA and covGT are covariance matrices with respect to the tracking area and are calculated by our method and artificially marked.Consider the following:(41)BD=18(meanA-meanGT)T[covA-covGT2]-1×(meanA-meanGT)+12ln|(covA+covGT)/2||covA|1/2|covGT|1/2.The Bhattacharyya distance between the tracked object area and the artificial marked region is shown in Figure6. Among nine case studies, our proposed method has shown a marked improvement on the tracking accuracy as compared with the two existing trackers (Tracker-1 and Tracker-2). It is mainly due to our combination of perceptual spatial-space generation model and the enhanced mean shift. The averaged Bhattacharyya distance on different video is shown in Table 3. Our proposed method has the smallest average tracking deviation between different methods.Table 3
Averaged Bhattacharyya distances for our proposed method,Tracker-1, and Tracker-2.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
1.2965
1.2891
0.5669
Satellite-2
1.1051
1.1902
0.4978
Satellite-3
1.2239
1.2536
0.5564
Automobile-1
0.8567
0.8944
0.4523
Running
0.9812
0.8845
0.4808
Automobile-2
1.8749
1.9861
0.7126
Highway
0.7746
0.7983
0.4533
Walking
1.7622
1.7906
0.6854
Automobile-3
1.7170
1.7699
1.0247
Walking-2
1.6012
1.6204
0.8040Figure 6
Bhattacharyya distances between the tracking method and the artificial markers. Red curve: our proposed method; blue curve:Tracker-1 in [27]; green curve: Tracker-2 in [28].


## 4.1. Contour Evolution in Prototype Pyramid

Contour evolution experimental results are shown in Figure2. Each prototype pyramid layer is calculated independently. The performances have shown that the contour evolution has a better continuity in the layer-by-layer prototype pyramid, and the approximate effects are derived from a perceptual spatial-space generation model and are closer to the human visual perception system.Contour evolution on spatial video sequences. (a) Original video frame and (b)–(d) contour evolution from a coarse to fine scale.
(a)
(b)
(c)
(d)


## 4.2. Markov Random Field Representation

The Markov random field representation is shown in Figure3. The significant representative region derivate from the perceptual spatial-space generation model contains significant feature information, which is closely related to the different color and texture distribution in the spatial area. The experimental results show that the region representation effect, which has been smoothed, could highlight the sensitivity of the motion area more than the initial representation.Markov random field representation in the perceptual spatial-space generation model. (a) Original frame and (b)–(c) initial and global smoothing representation.
(a)
(b)
(c)


## 4.3. Enhanced Mean Shift Object Tracking

We conducted experiments on the ten video sequences, in which the tracking objects include spatial satellite and aircraft, highway and park surveillance, and the human body. The enhanced mean shift conducted an iterative calculation 20 times on each video sequence. The normalized matrix bandwidth parameters are determined by a different experimental sample. In our experiments, it is 0.63 for video sequences 1, 2, and 3, 0.42 for video sequences 4 and 6, 0.56 for video sequences 5 and 8, 0.21 for video sequence 7,  and 0.60 for video sequence 9 and 10.


### 4.3.1. Satellite-1, 2, and 3

Tracker-1: the particle filter will have a negative impact on the horizontal direction, and the motion estimation will appear noncontinuous. Tracker-2: the distance metric learning will affect motion area determination, and the rectangular window tracking will produce some deviation. Our proposed algorithm can better track the spatial objects, and satellites and aircraft can be completely contained in the rectangular window with a similar color distribution and background quiver. The edge deviation variance can be controlled well using spatial object detection, with no offset and blurring.


### 4.3.2. Automobile-1 and 2

In a nighttime environment, as the weak light and brightness, Tracker-1 and Tracker-2 obtain a vague tracking result, and the confusion area between background and the tracking object becomes larger. In the Automobile-2 sequence in particular, as the illumination from other automotive foreground lamps, Track-1 produces particularly serious deviation. Our proposed enhanced mean shift method could distinguish the tracking object from background confusion, and the tracking results do not appear to obviously deviate.


### 4.3.3. Highway

Tracker-1: this method will lose the tracking center in some frames. There are some cross-rectangular windows between the far and close vision sequences. Shape errors exist in the rectangle window estimations. Tracker-2: the tracking results also have a tracking deviation in the far vision. Our proposed method can maintain consistency between different visions and does not appear to have huge deviations.


### 4.3.4. Running, Walking

There are no obvious differences between the different methods. Except for some slow motion (walking body), Tracker-2 shows some partial deviations.


### 4.3.5. Automobile-3, Walking-2

These cases are used to further test the robustness of our proposed method in the scene containing two or more moving objects. We could observe that Tracker-2 results in less accurate boxes probably due to its poor estimate of different moving object center position in the same scene. The performance of Tracker-1 is somewhat better; however, it also produces partial deviations. Our proposed method is more robust when dealing with scenarios of two or more moving objects.


## 4.3.1. Satellite-1, 2, and 3

Tracker-1: the particle filter will have a negative impact on the horizontal direction, and the motion estimation will appear noncontinuous. Tracker-2: the distance metric learning will affect motion area determination, and the rectangular window tracking will produce some deviation. Our proposed algorithm can better track the spatial objects, and satellites and aircraft can be completely contained in the rectangular window with a similar color distribution and background quiver. The edge deviation variance can be controlled well using spatial object detection, with no offset and blurring.


## 4.3.2. Automobile-1 and 2

In a nighttime environment, as the weak light and brightness, Tracker-1 and Tracker-2 obtain a vague tracking result, and the confusion area between background and the tracking object becomes larger. In the Automobile-2 sequence in particular, as the illumination from other automotive foreground lamps, Track-1 produces particularly serious deviation. Our proposed enhanced mean shift method could distinguish the tracking object from background confusion, and the tracking results do not appear to obviously deviate.


## 4.3.3. Highway

Tracker-1: this method will lose the tracking center in some frames. There are some cross-rectangular windows between the far and close vision sequences. Shape errors exist in the rectangle window estimations. Tracker-2: the tracking results also have a tracking deviation in the far vision. Our proposed method can maintain consistency between different visions and does not appear to have huge deviations.


## 4.3.4. Running, Walking

There are no obvious differences between the different methods. Except for some slow motion (walking body), Tracker-2 shows some partial deviations.


## 4.3.5. Automobile-3, Walking-2

These cases are used to further test the robustness of our proposed method in the scene containing two or more moving objects. We could observe that Tracker-2 results in less accurate boxes probably due to its poor estimate of different moving object center position in the same scene. The performance of Tracker-1 is somewhat better; however, it also produces partial deviations. Our proposed method is more robust when dealing with scenarios of two or more moving objects.


## 4.4. Objective Evaluation

We use three objective evaluations for evaluating the object tracking performance (Figure4).Figure 4
Experimental results from the tracking, marked by the bounding box. The red (solid line) box is from our proposed method, the blue (dashed line) box is Tracker-1 from the particle filters in [27], and the green (dashed line) box is Tracker-2 from the online distance metric learning in [28]. The rows 1–10 contain images from the videos “Satellite-1,” “Satellite-2,” “Satellite-3,” “Automobile-1,” “Running,” “Automobile-2,” “Highway,” “Walking,” “Automobile-3” and “Walking-2,” respectively.


### 4.4.1. Euclidian Distance

The Euclidian distance is the distance between rectangular windows obtained by tracking methods and the artificially marked. The specific calculation is as follows:(39)D=14∑i=14(xA,i-xGT,i)2+(yA,i-yGT,i)2,where(xA,i,yA,i), (xGT,i,yGT,i), and  i=1,2,3,4 are corner coordinates of the rectangular window calculated by tracking methods and artificially marked. Figure 5 shows the Euclidian distance between the tracked and artificially marked areas for our proposed method and the two other trackers on the videos.Figure 5
Euclidian distances between the tracked and artificially marked areas. Red curve: our proposed method; blue curve:Tracker-1 in [27]; green curve: Tracker-2 in [28].The averaged Euclidian distance calculated on all videos is shown in Table1. Compared to the distance values from Tracker-1 and Tracker-2, our proposed method clearly shows smaller and bounded Euclidian distances for the tested videos.Table 1
Results of the averaged Euclidian distance over all frames in each video.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
38.41
39.60
17.82
Satellite-2
60.97
60.44
23.45
Satellite-3
65.41
56.37
23.32
Automobile-1
27.64
28.22
14.23
Running
24.53
25.76
16.44
Automobile-2
52.67
59.21
32.87
Highway
22.16
21.06
10.54
Walking
25.29
23.19
12.73
Automobile-3
36.53
36.47
25.71
Walking-2
33.46
33.43
23.42


### 4.4.2. Mean Square Errors (MSEs)

We have(40)MSE=1N∑i=1N(xiA-xiGT)2+(yiA-yiGT)2,where(xiA,yiA), (xiGT,yiGT) is the center of the tracking area obtained by methods and artificially marked, respectively. N is the total number of video sequence frames. The experimental results are shown in Table 2 and, it can be seen that our proposed method has the minimum MSE, which means it has the lowest tracking deviation. Our method has obvious advantages compared to other methods.Table 2
Results of averaged MSE errors for the tracked box from our proposed method,Tracker-1, and Tracker-2.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
2.7865
6.2853
1.5866
Satellite-2
5.2368
13.4572
3.2914
Satellite-3
15.7326
12.3574
5.1233
Automobile-1
6.2357
7.8211
2.5671
Running
8.4596
7.7324
1.9635
Automobile-2
14.3687
18.5964
4.2158
Highway
4.2365
3.2151
1.1623
Walking
7.5642
9.2534
3.1486
Automobile-3
9.7749
10.1037
4.3357
Walking-2
8.9681
9.6614
4.0654


### 4.4.3. Bhattacharyya Distance

The Bhattacharyya distance is used to judge the deviation degree between the tracking area and the actual motion area. The specific calculation method is as follows:meanA and meanGT are the mean vectors with respect to the tracking area and are calculated by our method and artificially marked. The variables covA and covGT are covariance matrices with respect to the tracking area and are calculated by our method and artificially marked.Consider the following:(41)BD=18(meanA-meanGT)T[covA-covGT2]-1×(meanA-meanGT)+12ln|(covA+covGT)/2||covA|1/2|covGT|1/2.The Bhattacharyya distance between the tracked object area and the artificial marked region is shown in Figure6. Among nine case studies, our proposed method has shown a marked improvement on the tracking accuracy as compared with the two existing trackers (Tracker-1 and Tracker-2). It is mainly due to our combination of perceptual spatial-space generation model and the enhanced mean shift. The averaged Bhattacharyya distance on different video is shown in Table 3. Our proposed method has the smallest average tracking deviation between different methods.Table 3
Averaged Bhattacharyya distances for our proposed method,Tracker-1, and Tracker-2.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
1.2965
1.2891
0.5669
Satellite-2
1.1051
1.1902
0.4978
Satellite-3
1.2239
1.2536
0.5564
Automobile-1
0.8567
0.8944
0.4523
Running
0.9812
0.8845
0.4808
Automobile-2
1.8749
1.9861
0.7126
Highway
0.7746
0.7983
0.4533
Walking
1.7622
1.7906
0.6854
Automobile-3
1.7170
1.7699
1.0247
Walking-2
1.6012
1.6204
0.8040Figure 6
Bhattacharyya distances between the tracking method and the artificial markers. Red curve: our proposed method; blue curve:Tracker-1 in [27]; green curve: Tracker-2 in [28].


## 4.4.1. Euclidian Distance

The Euclidian distance is the distance between rectangular windows obtained by tracking methods and the artificially marked. The specific calculation is as follows:(39)D=14∑i=14(xA,i-xGT,i)2+(yA,i-yGT,i)2,where(xA,i,yA,i), (xGT,i,yGT,i), and  i=1,2,3,4 are corner coordinates of the rectangular window calculated by tracking methods and artificially marked. Figure 5 shows the Euclidian distance between the tracked and artificially marked areas for our proposed method and the two other trackers on the videos.Figure 5
Euclidian distances between the tracked and artificially marked areas. Red curve: our proposed method; blue curve:Tracker-1 in [27]; green curve: Tracker-2 in [28].The averaged Euclidian distance calculated on all videos is shown in Table1. Compared to the distance values from Tracker-1 and Tracker-2, our proposed method clearly shows smaller and bounded Euclidian distances for the tested videos.Table 1
Results of the averaged Euclidian distance over all frames in each video.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
38.41
39.60
17.82
Satellite-2
60.97
60.44
23.45
Satellite-3
65.41
56.37
23.32
Automobile-1
27.64
28.22
14.23
Running
24.53
25.76
16.44
Automobile-2
52.67
59.21
32.87
Highway
22.16
21.06
10.54
Walking
25.29
23.19
12.73
Automobile-3
36.53
36.47
25.71
Walking-2
33.46
33.43
23.42


## 4.4.2. Mean Square Errors (MSEs)

We have(40)MSE=1N∑i=1N(xiA-xiGT)2+(yiA-yiGT)2,where(xiA,yiA), (xiGT,yiGT) is the center of the tracking area obtained by methods and artificially marked, respectively. N is the total number of video sequence frames. The experimental results are shown in Table 2 and, it can be seen that our proposed method has the minimum MSE, which means it has the lowest tracking deviation. Our method has obvious advantages compared to other methods.Table 2
Results of averaged MSE errors for the tracked box from our proposed method,Tracker-1, and Tracker-2.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
2.7865
6.2853
1.5866
Satellite-2
5.2368
13.4572
3.2914
Satellite-3
15.7326
12.3574
5.1233
Automobile-1
6.2357
7.8211
2.5671
Running
8.4596
7.7324
1.9635
Automobile-2
14.3687
18.5964
4.2158
Highway
4.2365
3.2151
1.1623
Walking
7.5642
9.2534
3.1486
Automobile-3
9.7749
10.1037
4.3357
Walking-2
8.9681
9.6614
4.0654


## 4.4.3. Bhattacharyya Distance

The Bhattacharyya distance is used to judge the deviation degree between the tracking area and the actual motion area. The specific calculation method is as follows:meanA and meanGT are the mean vectors with respect to the tracking area and are calculated by our method and artificially marked. The variables covA and covGT are covariance matrices with respect to the tracking area and are calculated by our method and artificially marked.Consider the following:(41)BD=18(meanA-meanGT)T[covA-covGT2]-1×(meanA-meanGT)+12ln|(covA+covGT)/2||covA|1/2|covGT|1/2.The Bhattacharyya distance between the tracked object area and the artificial marked region is shown in Figure6. Among nine case studies, our proposed method has shown a marked improvement on the tracking accuracy as compared with the two existing trackers (Tracker-1 and Tracker-2). It is mainly due to our combination of perceptual spatial-space generation model and the enhanced mean shift. The averaged Bhattacharyya distance on different video is shown in Table 3. Our proposed method has the smallest average tracking deviation between different methods.Table 3
Averaged Bhattacharyya distances for our proposed method,Tracker-1, and Tracker-2.
Video
Tracker-1
Tracker-2
Proposed
Satellite-1
1.2965
1.2891
0.5669
Satellite-2
1.1051
1.1902
0.4978
Satellite-3
1.2239
1.2536
0.5564
Automobile-1
0.8567
0.8944
0.4523
Running
0.9812
0.8845
0.4808
Automobile-2
1.8749
1.9861
0.7126
Highway
0.7746
0.7983
0.4533
Walking
1.7622
1.7906
0.6854
Automobile-3
1.7170
1.7699
1.0247
Walking-2
1.6012
1.6204
0.8040Figure 6
Bhattacharyya distances between the tracking method and the artificial markers. Red curve: our proposed method; blue curve:Tracker-1 in [27]; green curve: Tracker-2 in [28].


## 5. Conclusions

In this paper, we propose a new intelligent modeling method using the enhanced mean shift method based on a perceptual spatial-space model for spatial object tracking. The perceptual spatial-space model can obtain a continuous spatial object contour and highlight tracking object saliency. Enhanced mean shift method uses enhanced version mean shift, which focuses on the estimation of spatial shape parameters. The method could effectively cope with severe spatial interferences. The comparison between our method and other state-of-the-art methods demonstrates that our proposed method has a higher tracking accuracy and precision. In future research, we can incorporate more spatial object information, such as spatial textures and aircraft shapes, into our intelligent model to generate a more robust spatial object tracking method.



---

*Source: 420286-2013-04-11.xml*