# Moving Object Detection for Dynamic Background Scenes Based on Spatiotemporal Model

Authors: Yizhong Yang; Qiang Zhang; Pengfei Wang; Xionglou Hu; Nengju Wu
Journal: Advances in Multimedia
 (2017)

---

## Abstract

Moving object detection in video streams is the first step of many computer vision applications. Background modeling and subtraction for moving detection is the most common technique for detecting, while how to detect moving objects correctly is still a challenge. Some methods initialize the background model at each pixel in the firstN frames. However, it cannot perform well in dynamic background scenes since the background model only contains temporal features. Herein, a novel pixelwise and nonparametric moving object detection method is proposed, which contains both spatial and temporal features. The proposed method can accurately detect the dynamic background. Additionally, several new mechanisms are also proposed to maintain and update the background model. The experimental results based on image sequences in public datasets show that the proposed method provides the robustness and effectiveness in dynamic background scenes compared with the existing methods.


---

## Body

## 1. Introduction

Recently, background modeling and subtraction became the most popular technique for moving object detection in computer vision, such as object recognition and traffic surveillance [1–9].Compared to optical flow [10, 11] and interframe difference algorithms [12], background subtraction algorithm needs less computation and performs better, and it is more flexible and effective. The idea of background subtraction is to differentiate the current image from a reference background model. These algorithms initialize a background model at first to represent the scene with no moving objects and then detect the moving objects by computing the difference between the current frame and the background model. Dynamic background is a challenge for background subtraction, such as waving tree leaves and ripples on river. In the past several years, many background subtraction algorithms have been proposed, and most of them focus on building more effective background model to handle dynamic background as follows:(1)
Features: texture and color [13–15](2)
Combining methods: combining two or more background models as the new model [16](3)
Updating the background model [17]In this paper, a new pixelwise and nonparametric moving object detection method is proposed. Background model is built by the firstN1 frames and sampling m times in 3 × 3 neighborhood region randomly. On the one hand, spatiotemporal model represents dynamic background scenes well. On the other hand, a new update strategy makes the background model fit the dynamic background. In addition, the proposed method can deal with ghost well. Experimental results show that the proposed method can efficiently and correctly detect the moving objects from the dynamic background.This paper is organized as follows. In the next section, an overview of existing approaches of background subtraction is presented. Section3 describes the proposed method in detail, and then Section 4 provides the experimental results and comparison with other methods. Section 5 includes conclusions and further research directions.


## 2. Related Work

In this section, some background subtraction methods will be introduced, which are divided into parametric and nonparametric models.For parametric models, the most commonly used method is Gaussian Mixture Model (GMM) [18]. Before GMM, a per-pixel Gaussian model was proposed [19], which calculated the mean and standard deviation for each pixel at first and then compared the probability with a certain threshold of each pixel to classify the current pixel as background or foreground. But this Gaussian model cannot deal with noise and dynamic situation. GMM was proposed to solve these problems. GMM usually set three-to-five Gaussian models for each pixel and updated the model after matching. Several papers [20, 21] improved the GMM method to be more flexible and efficient in recent years.In contrast to parametric models, nonparametric models are commonly set up by the collection of the observed pixel values or neighborhood pixel values of each pixel. Kernel Density Estimation (KDE) [22] was proposed to open the door of hot research of nonparametric methods. In [13], a clustering technique was proposed to set up a nonparametric background model. The background model’s samples of each pixel were clustered into the set of code words. In [23], Wang et al. chose to include large number (up to 200) of samples in the background model. Since the background models set up by [13, 23] are only based on temporal information, they cannot deal with dynamic background scenes well without the spatial information. In ViBe [24, 25], a random scheme was introduced to set up and update background models. They initialized the background model from the first frame, and the model elements were sampled from the collection of each pixel’s neighborhood randomly. ViBe shows robustness and effectiveness for dynamic background scenes in a sense. In order to improve ViBe further, Hofmann et al. [17] proposed an adaptive scheme to automatically tune the decision threshold based on previous decisions made by the system. However, the background models set up by [17, 24, 25] are only based on spatial information. The lack of temporal information makes it hard to deal with time-related situation well. In [26], a modified Local Binary Similarity Pattern (LBSP) descriptor was proposed to set up the background model in feature space. It calculated the LBSP descriptor by absolute difference which is different from LBP. What is more, intra-LSBP and inter-LSBP were calculated in the same predetermined pattern to capture both texture and intensity changes. The change detection results from LSBP proved efficiency against many complex algorithms. Reference [27] improved LSBP in threshold area and combined with ViBe method to detect motion. The improvement was obviously in noisy and blurred regions. Reference [28] proposed spatiotemporal background model by integrating the concepts of a local feature-based approach and a statistical approach into a single framework; the results show that it can deal with illumination and dynamic background scenes well. These algorithms contain both temporal information and spatial information, resulting in not bad performance.Initialization and update strategy are important steps common for background modeling. As for initialization, some background subtraction methods initialized the background models with pixel values at each pixel in the firstN frames [16]. However, it was not effective for dynamic background situation because of the lack of neighboring pixel information. Reference [24] initialized from the first frame by choosing the neighborhood pixel values as sample randomly. However, it initialized the background model by only one frame. In addition, it sampled 20 pixels as the background model in the field of current pixel neighborhood. However, there were only 8 pixels in neighborhood, which inevitability resulted in repeated selection. Then it would affect segmentation decision because of the ill-considered model. Reference [29] proposed a different method to initialize the background model. Every element of the model contained pixel value and an efficacy Ck, and the element with the least value of Ck will be removed or updated. However, element with the least value of Ck might not be the worst element in dynamic background scenes. As for update strategy, in [24], when a pixel has been classified as background, a random process determined whether this pixel was used to update the corresponding pixel model. It was workable but too blind to update the model well.Herein, a nonparametric model collecting both the history and the neighborhood pixel values is presented to improve the performance for dynamic background scenes. The proposed method, based on spatiotemporal model, collects pixel values as sample from the history and neighborhood of a pixel, and the model elements are sampled from neighborhood region in the firstN1 frames. As for update strategy, the range of randomness is decreased to increase the accuracy. All above methods proposed are different from other methods based on spatiotemporal model.


## 3. Spatiotemporal Model for Background

Normally, a background model can fit only one kind of scenes and it is difficult to get a universal background model which can deal with all the complex and diverse scenes. Some background subtraction methods combine the different models or features like texture together to get universal models. These methods regard every frame as the most complex scenes and result in a large amount of calculation. As for this question, this paper proposes a novel and simple method to model background for dynamic background scenes, and the idea is employed to initialize the model. Next, the details of our spatiotemporal model will be introduced. The diagram of the proposed method is shown in Figure1.Figure 1
Diagram of the proposed method.


### 3.1. Initialization

The proposed method initializes background model from the firstN1 frames. First of all, the spatial model BN(xi) can be initialized by picking out pixel value randomly in the neighborhood of xi for m times at each frame, and m is less than 8. (1)BN1xi=I1xi,I2xi,…,ImxiBN2xi=Im+1xi,Im+2xi,…,I2mxiBNN1xi=IN1-1×m+1xi,IN1-1×m+2xi,…,IN1×mxi.Then these spatial background models are integrated together to construct spatiotemporal modelBxi:(2)Bxi=BN1xi,BN2xi,…,BNN1xi.For the convenience of record,(3)Bxi=I1,I2,I3,…,IN,N=N1×m.As for the value ofN1, N, m will be discussed in Section 4 later. The spatial information and the temporal information are integrated, and the combined idea is used here without large amount of computation. The proposed background model is proved to be effective.


### 3.2. Segmentation Decision

Since the proposed model only consists of grayscale value of pixel, the segmentation decision is simple in our single model. It just compares the distance between the current pixel and the pixel in the background model, and the formula is shown as follows:(4)Fxi=1#distIxi,Bkxi<Rxi<#min0else,where Bkxi represents the kth element in model Bxi. #min defines the least number of elements in background model meeting the threshold condition. If Fxi=1, it implies that the pixel belongs to foreground, and conversely, the pixel belongs to background.


### 3.3. Updating Process

Background changes all the time in dynamic background scenes, so it is necessary to update the background model regularly to fit the dynamic background. In this section, update of the spatiotemporal model and adaptive update of decision threshold will be described in detail.


#### 3.3.1. Update of the Spatiotemporal Model

The proposed method divides the model elements into two parts, high-efficacy part and low-efficacy part. The elements which meet the formuladistIxi,Bkxi<Rxi belong to high-efficacy part, and the rest belong to low-efficacy part. Then the random strategy will be conducted in the range of these elements belonging to low-efficacy part. What is more, learning rate T is determined by experiments to fit the proposed method better.


#### 3.3.2. Update of the Neighborhood

Background pixels always exist together in some regions, so the neighborhood of a pixel may be background pixels if this pixel has been classified as background. However, it may not be true in the edge region. In conclusion, pixels in neighborhood region of a background pixel are more likely to be background pixels compared with other pixels. So the background model of neighborhood pixel will be updated as well with the same method introduced in Section3.3.1. After the update process, parameter #min will become #min-1 when segmentation decision is conducted in neighborhood, which is just like adaptive update.The update method above is a memoryless update strategy. The samples in the background model at timet are preserved after the update of the pixel model with the probability (N-1)/N. For any further time t+dt, this probability formula is shown as follows:(5)Pt,t+dt=N-1Nt+dt-t.This formula can also be written as follows:(6)Pt,t+dt=e-ln⁡N/N-1dt,where Pt,t+dt denotes the probability after time dt, and it shows that the expected remaining lifespan of any sample value of the model decays exponentially.


## 3.1. Initialization

The proposed method initializes background model from the firstN1 frames. First of all, the spatial model BN(xi) can be initialized by picking out pixel value randomly in the neighborhood of xi for m times at each frame, and m is less than 8. (1)BN1xi=I1xi,I2xi,…,ImxiBN2xi=Im+1xi,Im+2xi,…,I2mxiBNN1xi=IN1-1×m+1xi,IN1-1×m+2xi,…,IN1×mxi.Then these spatial background models are integrated together to construct spatiotemporal modelBxi:(2)Bxi=BN1xi,BN2xi,…,BNN1xi.For the convenience of record,(3)Bxi=I1,I2,I3,…,IN,N=N1×m.As for the value ofN1, N, m will be discussed in Section 4 later. The spatial information and the temporal information are integrated, and the combined idea is used here without large amount of computation. The proposed background model is proved to be effective.


## 3.2. Segmentation Decision

Since the proposed model only consists of grayscale value of pixel, the segmentation decision is simple in our single model. It just compares the distance between the current pixel and the pixel in the background model, and the formula is shown as follows:(4)Fxi=1#distIxi,Bkxi<Rxi<#min0else,where Bkxi represents the kth element in model Bxi. #min defines the least number of elements in background model meeting the threshold condition. If Fxi=1, it implies that the pixel belongs to foreground, and conversely, the pixel belongs to background.


## 3.3. Updating Process

Background changes all the time in dynamic background scenes, so it is necessary to update the background model regularly to fit the dynamic background. In this section, update of the spatiotemporal model and adaptive update of decision threshold will be described in detail.


### 3.3.1. Update of the Spatiotemporal Model

The proposed method divides the model elements into two parts, high-efficacy part and low-efficacy part. The elements which meet the formuladistIxi,Bkxi<Rxi belong to high-efficacy part, and the rest belong to low-efficacy part. Then the random strategy will be conducted in the range of these elements belonging to low-efficacy part. What is more, learning rate T is determined by experiments to fit the proposed method better.


### 3.3.2. Update of the Neighborhood

Background pixels always exist together in some regions, so the neighborhood of a pixel may be background pixels if this pixel has been classified as background. However, it may not be true in the edge region. In conclusion, pixels in neighborhood region of a background pixel are more likely to be background pixels compared with other pixels. So the background model of neighborhood pixel will be updated as well with the same method introduced in Section3.3.1. After the update process, parameter #min will become #min-1 when segmentation decision is conducted in neighborhood, which is just like adaptive update.The update method above is a memoryless update strategy. The samples in the background model at timet are preserved after the update of the pixel model with the probability (N-1)/N. For any further time t+dt, this probability formula is shown as follows:(5)Pt,t+dt=N-1Nt+dt-t.This formula can also be written as follows:(6)Pt,t+dt=e-ln⁡N/N-1dt,where Pt,t+dt denotes the probability after time dt, and it shows that the expected remaining lifespan of any sample value of the model decays exponentially.


## 3.3.1. Update of the Spatiotemporal Model

The proposed method divides the model elements into two parts, high-efficacy part and low-efficacy part. The elements which meet the formuladistIxi,Bkxi<Rxi belong to high-efficacy part, and the rest belong to low-efficacy part. Then the random strategy will be conducted in the range of these elements belonging to low-efficacy part. What is more, learning rate T is determined by experiments to fit the proposed method better.


## 3.3.2. Update of the Neighborhood

Background pixels always exist together in some regions, so the neighborhood of a pixel may be background pixels if this pixel has been classified as background. However, it may not be true in the edge region. In conclusion, pixels in neighborhood region of a background pixel are more likely to be background pixels compared with other pixels. So the background model of neighborhood pixel will be updated as well with the same method introduced in Section3.3.1. After the update process, parameter #min will become #min-1 when segmentation decision is conducted in neighborhood, which is just like adaptive update.The update method above is a memoryless update strategy. The samples in the background model at timet are preserved after the update of the pixel model with the probability (N-1)/N. For any further time t+dt, this probability formula is shown as follows:(5)Pt,t+dt=N-1Nt+dt-t.This formula can also be written as follows:(6)Pt,t+dt=e-ln⁡N/N-1dt,where Pt,t+dt denotes the probability after time dt, and it shows that the expected remaining lifespan of any sample value of the model decays exponentially.


## 4. Experiments and Results

In this section, a series of experiments are conducted to analyze the parameter setting and evaluated the performance of the proposed method with others. Here, we first express our gratitude to changedetection.net [34], which provides the datasets for our experiments. The datasets include six test videos on the category of dynamic background and several objective indexes for evaluating performance quantitatively:(7)Recall=TPTP+FNPrecision=TPTP+FPF-Measure=2×Precision×RecallPrecision+RecallFPR=FPFN+FPFNR=FNTP+FNPWC=100×FN+FPTP+FN+FP+TN,where True Positive (TP) is the number of correctly classified foreground pixels and True Negative (TN) is the number of correctly classified background pixels. On the other hand, False Positive (FP) is the number of background pixels that is incorrectly classified as foreground and False Negative (FN) is the number of foreground pixels that is incorrectly classified as background pixel in background subtraction method. The data above are used to calculate Recall, Precision, and F-Measure. Recall represents the percent of the correctly detected foreground relative to the ground truth foreground. Precision represents the percent of the correctly detected foreground relative to the detected foreground including true foreground and false foreground. F-Measure is a comprehensive index of Recall and Precision, which is primarily used to evaluate the performance of different parameters and different methods.The proposed method is implemented in C++ programming language with opencv2.4.9 on a core i3 CPU with 3.0 GHz and 2 G RAM.


### 4.1. Parameter Setting

It was mentioned in Section3 that we initialized the model from N1 frames and sampled elements from neighborhood randomly m times. We conducted a series of experiments on the adjustment of m and N1 with the fixed parameter, learning rate T and #min, and without postprocessing.It is clear that performance with parameterm from 5 to 6 and N1 from 6 to 10 are better in Figure 2. Further experiments tested with different parameters are shown in Table 1. Performance with different T value is shown in Figure 3.Table 1
Further experiments to chooseN1 and m.
F-M
N
1
6
7
8
9
10
m
5
0.7720
0.7855
0.7914
0.7788
0.7878
6
0.7844
0.7736
0.7822
0.7765
0.7739
N
1
=
8 and m=5 are the best choices, and N=N1×m=40 is also a desired parameter for small computational burden.Figure 2
Performance with differentN1 and m.Figure 3
Performance with differentT value.The parametersT and #min will be determined by experiments with fixed N1 and m. The experiment result of selecting T can be seen in Figure 3 and the experiment result of selecting #min can be seen in Figure 4.Figure 4
Performance with different#min.After these kinds of experiments (Figures3 and 4), the parameters were set as follows:(1)
N1=8, m=5, N=40.(2)
T=16.(3)
#min=5, R=20.(4)
A median filter step was applied, and it can be seen that, in Table2, a 9 × 9 window behaves better. The median filter is a step to make the results better, while, compared with other algorithm, this step is removed.Table 2
Performance of proposed method with postprocessing.
Size
3
×
3
5
×
5
7
×
7
9
×
9
11
×
11
F-Measure
0.8690
0.8813
0.8879
0.8888
0.8881


### 4.2. Comparison with Other Methods

Figures5(b) and 5(c) show the detection results of [13] and the proposed method from the input frame (a), respectively. The waving tree leaves in (a) are the dynamic background. Since [13] is a temporal-only model method, the background model lacks the neighborhood pixel information, which will regard the dynamic background as moving objects. The proposed method considers both temporal information and spatial information, setting up the background model from the first 8 frames and sampling 5 times in the 3 × 3 neighborhood region randomly. Therefore, the performance in dynamic background scenes is better than [13].Figure 5
Comparison of the dynamic situation. (a) The thirteenth frame of “Overpass” video. (b) The detection result of [13]. (c) The detection result of the proposed method.
(a)
(b)
(c)Figure6 shows the detection results of ViBe [24] and the proposed method. Since ViBe [24] sets up the background model only based on spatial information, time-related situation such as ghost may exist. As shown in Figure 6(c), it sets up background model just from the first frame and regards all pixels in it as background pixels without moving objects. If there are some moving objects in first frame and the moving objects move away (the fiftieth frame (b)), they will be detected as ghosts (cars marked in red rectangles in (c)). The background model of the proposed method contains not only spatial information but also temporal information, so it can recognize the moving objects from first frame. Therefore, ghost can be well eliminated.Figure 6
Comparison of ghost elimination. (a) and (b): the first and fiftieth frames of “Highway” video, respectively. (c) The detection result of the fiftieth frame by ViBe. (d) The detection result of the fiftieth frame by the proposed method.
(a)
(b)
(c)
(d)The proposed method focuses on building and updating more effective background model to deal with dynamic background scenes. The public dynamic background video datasets from changedetection.net, which are “Boats” with 7999 frames, “Canoe” with 1189 frames, “Fall” with 4000 frames, “Fountain01” with 1184 frames and “Fountain02” with 1499 frames, “Overpass” with 3000 frames, are used to conduct the experiments. For fair comparison, the results of the proposed method do not use any postprocessing. ViBe [24] and CodeBook [13] are two classical methods for background segmentation, so we conducted comparison between the proposed method and them. Experimental results are shown in Figure 7.Figure 7
Comparison of the detection results. (a) Input frames of the six videos of “dynamic background” in changedetection.net, and they are the 2000th frame in “Boats,” the 955th frame in Canoe, the 1892th frame in Fall, the 1147th frame in “Fountain01,” the 745th frame in “Fountain02,” and the 2462th frame in “Overpass” from top to bottom. (b) Ground truth of (a). (c) Results of ViBe. (d) Results of CodeBook. (e) Results of the proposed method.
(a)
(b)
(c)
(d)
(e)Beyond dynamic background scenes, the results of other categories in changedetection.net are shown in Figure8. It can be seen that the proposed method performs well in several different categories, such as “Bad Weather,” “Baseline”, “Thermal,” and “Intermittent Object Motion.” But in other categories, the proposed method performs not very well. For example, in “PTZ” category, after the camera moves, the proposed method needs a rather long time to learn the new background by the update process, which may result in false detection during this process. However, although the proposed method is not a universal method, it can deal with most scenes satisfactorily.Figure 8
Detection results of other categories. (a) and (c): input test video; (b) and (d): detection results. The first row is the “Bad Weather” category, the second row is the “Baseline” category, the third row is the “Thermal” category, the fourth row is “Intermittent Object Motion” category and “Turbulence” category, the fifth row is “Low Framerate” category and “Night Videos” category, and the sixth row is “Camera Jitter” category and “PTZ” category.
(a)
(b)
(c)
(d)The quantitative comparison results of “dynamic background” category between the proposed method and more other background subtraction methods are shown in Table3. Among these methods, ViBe [24] is a nonparameter algorithm, from which the proposed method is derived. LOBSTER [27] and Multiscale Spatiotemporal BG Model [30] are spatiotemporal background modeling algorithms, which are similar to the proposed method. EFIC [31] is a popular method in changedetection.net. TVRPCA [32] is an advanced RPCA based method, which is also designed for dynamic background scenes. As shown in Table 3, AAPSA [33] has the highest F-Measure for its autoadaptive strategy. Expect AASPA, in the aspect of F-Measure, the proposed method gets the highest score. Herein, although the proposed method’s F-Measure is not the highest, it can deal with not only dynamic background scenes well but also ghost elimination.Table 3
Comparison of performance between the proposed method and others.
Methods
Recall
FPR
FNR
PWC
PRE
F-Measure
ViBe [24]
0.7222
0.0104
0.2778
1.2796
0.5346
0.5652
LOBSTER [27]
0.7670
0.0180
0.0023
1.9984
0.5923
0.5679
Multiscale Spatiotemporal BG Model [30]
0.7392
0.0095
0.2608
1.1365
0.5515
0.5953
EFIC [31]
0.6667
0.0144
0.3333
0.9154
0.6849
0.5779
TVRPCA [32]
0.56
—
—
—
0.74
0.61
AAPSA [33]
0.6955
0.0011
0.3045
0.4992
0.7336
0.6706
Proposed method without postprocessing
0.6692
0.5322
0.3318
1.2762
0.6084
0.6213
Proposed method with postprocessing
0.7296
0.2773
0.2704
0.2800
0.8755
0.7960


## 4.1. Parameter Setting

It was mentioned in Section3 that we initialized the model from N1 frames and sampled elements from neighborhood randomly m times. We conducted a series of experiments on the adjustment of m and N1 with the fixed parameter, learning rate T and #min, and without postprocessing.It is clear that performance with parameterm from 5 to 6 and N1 from 6 to 10 are better in Figure 2. Further experiments tested with different parameters are shown in Table 1. Performance with different T value is shown in Figure 3.Table 1
Further experiments to chooseN1 and m.
F-M
N
1
6
7
8
9
10
m
5
0.7720
0.7855
0.7914
0.7788
0.7878
6
0.7844
0.7736
0.7822
0.7765
0.7739
N
1
=
8 and m=5 are the best choices, and N=N1×m=40 is also a desired parameter for small computational burden.Figure 2
Performance with differentN1 and m.Figure 3
Performance with differentT value.The parametersT and #min will be determined by experiments with fixed N1 and m. The experiment result of selecting T can be seen in Figure 3 and the experiment result of selecting #min can be seen in Figure 4.Figure 4
Performance with different#min.After these kinds of experiments (Figures3 and 4), the parameters were set as follows:(1)
N1=8, m=5, N=40.(2)
T=16.(3)
#min=5, R=20.(4)
A median filter step was applied, and it can be seen that, in Table2, a 9 × 9 window behaves better. The median filter is a step to make the results better, while, compared with other algorithm, this step is removed.Table 2
Performance of proposed method with postprocessing.
Size
3
×
3
5
×
5
7
×
7
9
×
9
11
×
11
F-Measure
0.8690
0.8813
0.8879
0.8888
0.8881


## 4.2. Comparison with Other Methods

Figures5(b) and 5(c) show the detection results of [13] and the proposed method from the input frame (a), respectively. The waving tree leaves in (a) are the dynamic background. Since [13] is a temporal-only model method, the background model lacks the neighborhood pixel information, which will regard the dynamic background as moving objects. The proposed method considers both temporal information and spatial information, setting up the background model from the first 8 frames and sampling 5 times in the 3 × 3 neighborhood region randomly. Therefore, the performance in dynamic background scenes is better than [13].Figure 5
Comparison of the dynamic situation. (a) The thirteenth frame of “Overpass” video. (b) The detection result of [13]. (c) The detection result of the proposed method.
(a)
(b)
(c)Figure6 shows the detection results of ViBe [24] and the proposed method. Since ViBe [24] sets up the background model only based on spatial information, time-related situation such as ghost may exist. As shown in Figure 6(c), it sets up background model just from the first frame and regards all pixels in it as background pixels without moving objects. If there are some moving objects in first frame and the moving objects move away (the fiftieth frame (b)), they will be detected as ghosts (cars marked in red rectangles in (c)). The background model of the proposed method contains not only spatial information but also temporal information, so it can recognize the moving objects from first frame. Therefore, ghost can be well eliminated.Figure 6
Comparison of ghost elimination. (a) and (b): the first and fiftieth frames of “Highway” video, respectively. (c) The detection result of the fiftieth frame by ViBe. (d) The detection result of the fiftieth frame by the proposed method.
(a)
(b)
(c)
(d)The proposed method focuses on building and updating more effective background model to deal with dynamic background scenes. The public dynamic background video datasets from changedetection.net, which are “Boats” with 7999 frames, “Canoe” with 1189 frames, “Fall” with 4000 frames, “Fountain01” with 1184 frames and “Fountain02” with 1499 frames, “Overpass” with 3000 frames, are used to conduct the experiments. For fair comparison, the results of the proposed method do not use any postprocessing. ViBe [24] and CodeBook [13] are two classical methods for background segmentation, so we conducted comparison between the proposed method and them. Experimental results are shown in Figure 7.Figure 7
Comparison of the detection results. (a) Input frames of the six videos of “dynamic background” in changedetection.net, and they are the 2000th frame in “Boats,” the 955th frame in Canoe, the 1892th frame in Fall, the 1147th frame in “Fountain01,” the 745th frame in “Fountain02,” and the 2462th frame in “Overpass” from top to bottom. (b) Ground truth of (a). (c) Results of ViBe. (d) Results of CodeBook. (e) Results of the proposed method.
(a)
(b)
(c)
(d)
(e)Beyond dynamic background scenes, the results of other categories in changedetection.net are shown in Figure8. It can be seen that the proposed method performs well in several different categories, such as “Bad Weather,” “Baseline”, “Thermal,” and “Intermittent Object Motion.” But in other categories, the proposed method performs not very well. For example, in “PTZ” category, after the camera moves, the proposed method needs a rather long time to learn the new background by the update process, which may result in false detection during this process. However, although the proposed method is not a universal method, it can deal with most scenes satisfactorily.Figure 8
Detection results of other categories. (a) and (c): input test video; (b) and (d): detection results. The first row is the “Bad Weather” category, the second row is the “Baseline” category, the third row is the “Thermal” category, the fourth row is “Intermittent Object Motion” category and “Turbulence” category, the fifth row is “Low Framerate” category and “Night Videos” category, and the sixth row is “Camera Jitter” category and “PTZ” category.
(a)
(b)
(c)
(d)The quantitative comparison results of “dynamic background” category between the proposed method and more other background subtraction methods are shown in Table3. Among these methods, ViBe [24] is a nonparameter algorithm, from which the proposed method is derived. LOBSTER [27] and Multiscale Spatiotemporal BG Model [30] are spatiotemporal background modeling algorithms, which are similar to the proposed method. EFIC [31] is a popular method in changedetection.net. TVRPCA [32] is an advanced RPCA based method, which is also designed for dynamic background scenes. As shown in Table 3, AAPSA [33] has the highest F-Measure for its autoadaptive strategy. Expect AASPA, in the aspect of F-Measure, the proposed method gets the highest score. Herein, although the proposed method’s F-Measure is not the highest, it can deal with not only dynamic background scenes well but also ghost elimination.Table 3
Comparison of performance between the proposed method and others.
Methods
Recall
FPR
FNR
PWC
PRE
F-Measure
ViBe [24]
0.7222
0.0104
0.2778
1.2796
0.5346
0.5652
LOBSTER [27]
0.7670
0.0180
0.0023
1.9984
0.5923
0.5679
Multiscale Spatiotemporal BG Model [30]
0.7392
0.0095
0.2608
1.1365
0.5515
0.5953
EFIC [31]
0.6667
0.0144
0.3333
0.9154
0.6849
0.5779
TVRPCA [32]
0.56
—
—
—
0.74
0.61
AAPSA [33]
0.6955
0.0011
0.3045
0.4992
0.7336
0.6706
Proposed method without postprocessing
0.6692
0.5322
0.3318
1.2762
0.6084
0.6213
Proposed method with postprocessing
0.7296
0.2773
0.2704
0.2800
0.8755
0.7960


## 5. Conclusion

In this paper, a novel change detection method of nonparametric background segmentation for dynamic background scenes is proposed. The background model is built by sampling 5 times in 3 × 3 neighborhood region randomly from first 8 frames. The samples of background model are separated to high-efficacy part and low-efficacy part, and the samples in low-efficacy part will be replaced randomly. The update strategy which replaces sample in low-efficacy part can continuously optimize the background model. It can be seen from the experimental results that the proposed method is robust in dynamic background scenes and ghost elimination compared to other methods.



---

*Source: 5179013-2017-06-18.xml*