<figure><img src="image_1.png" /></figure>

HAL
open science

# Motion Segmentation Based on Structure-Texture Decomposition and Improved Three Frame Differencing

Sandeep Singh Sengar

► To cite this version:

Sandeep Singh Sengar. Motion Segmentation Based on Structure-Texture Decomposition and Improved Three Frame Differencing. 15th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), May 2019, Heronissos, Greece. pp.609-622, 10.1007/978-3-030-19823-7_51. hal-02331299

HAL Id: hal-02331299

https://inria.hal.science/hal-02331299

Submitted on 24 Oct 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Motion segmentation based on structure-texture decomposition and improved three frame differencing*

Sandeep Singh Sengar¹[0000-0003-2171-9332]

Department of Computer Science and Engineering
SRM University-AP, Amaravati 522 502, India
sandeepsingh.s@srmap.edu.in

**Abstract.** Motion segmentation from the video datasets has several important applications like traffic monitoring, action recognition, visual object tracking, and video surveillance. The proposed technique combines the structure-texture decomposition and the improved three frames differencing for motion segmentation. First, the Osher and Vese approach is employed to decompose the video frame into two components, viz., structure and texture/noise. Now, to eliminate the noise, only the structure components are employed for further steps. Subsequently, the difference between (i) the current frame and the previous frame as well as (ii) the current frame and the next frame are estimated. Next, both the difference frames are combined using pixel-wise maximum operation. Each combined difference frame is then partitioned into non-overlapping blocks, and the intensity sum as well as median of each block is computed. Successively, target objects are detected with the help of threshold and intensity median. Finally, post-processing in the form of morphology operation and connected component analysis is carried out to accurately find the foreground. Our technique has been formulated, implemented and tested on publicly available standard benchmark datasets and it is proved from performance analysis that our technique exhibits efficient outcomes than existing approaches.

**Keywords:** Structure-texture decomposition · motion segmentation · frame difference · morphology · block.

## 1 Introduction

Motion segmentation is one of the basic and critical approaches of computer vision system. Motion segmentation technique is motivated by the assumption that foreground objects commonly involve changes in the intensity between successive frames. This approach is mandatory in a real environment and should be applicable to difficult circumstances, like background clutter, fake motion, and changes due to noise and illumination deviation. All of these situations lead to the false region detection for the moving targets.

Currently, background subtraction [1], optical flow [2] and frame differencing [3] are commonly used techniques for motion segmentation. The background subtraction technique depends on the accuracy of generated background; if the background is not

<footnote>* SRM University-AP, Amaravati.</footnote>
accurate then there will be large segmentation error. The optical flow technique is com-
plex and there are high amount of noise in detected result due to involvement of differ-
ential operations in it. On the other side, frame differencing is a simple and effective
technique to segmenting the moving objects by calculating the pixel-wise difference be-
tween successive frames. However, there are two most common problems in two frames
differencing viz., (i) background clutter (ii) foreground aperture. To reduce these prob-
lems, Sengar et al. [4] proposed an enhancement of frame differencing technique. This
is a straightforward method, which detects the moving objects by employing block
based three frame differencing. However, detection accuracy of this method is low for
some frames of video data due to the randomly generated noise. To improve the de-
tection accuracy, Sengar et al. [5] proposed another approach for motion segmentation
using block based optical flow. However, processing time of this work is high due to the
local processing. In order to improve the detection accuracy, reduce the false positives
and processing time, the structure-texture decomposition and three frame differencing
based motion detection approach is proposed. The proposed technique first extracts the
frames from the video datasets, followed by color to gray-scale processing. Next, the
structure component of the gray-scale video frame is extracted using structure-texture
decomposition technique so as to remove the texture/noise. Subsequently, block based
improved three frame differencing algorithm is employed on extracted structure compo-
nent for motion segmentation. Finally, post-processing steps using mathematical mor-
phology operation and connected component analysis are performed to accurately de-
tect the foreground. Our method is simple and has considerably low computational cost.
We have tested the proposed method on different publically available benchmark video
datasets and also compared with existing techniques. Our approach has higher detection
accuracy than widely used recent techniques.

As discussed earlier, there are certain problems available in existing motion seg-
mentation techniques. Here, in this approach we have reduced those. It is an important
technique for visual surveillance, traffic monitoring and other several related applica-
tions. This scheme can be the initial step for visual object tracking, where future position
of target can be predicted after detecting it in previous video frames.

The organization of remaining part of this paper is as follows. The survey on re-
lated methodologies are presented in Section 2. The proposed motion segmentation
method and it's implementation details are provided in Sections 3 and 4 respectively.
Experimental results with the qualitative as well as quantitative evaluations are shown
in section 5. Finally, conclusion of our technique is provided in Section 6.

## 2 Related work

Several schemes for motion segmentation have been presented by image processing
and computer vision groups to deal with challenges like ghosting, illumination vari-
ations, object pose variation, occlusion and out-of-plane rotation [6,7,8]. One of the
methods for textured image segmentation has been presented by Sandberg et al. [9].
This scheme has employed active contour model within Gabor filter environment. El-
harrouss et al. [1] proposed an approach for motion detection using the structure-texture
decomposition and the background subtraction. Block-wise low rank texture character-
ization is employed for cartoon-texture image decomposition model [10]. This model
works like a convex optimization problem. Surface normals and structure-texture de-
composition are used for intrinsic image decomposition [11]. High quality outcomes are
produced by this approach with less number of constraints. Wells et al. [12] presented
a work for image decomposition in the context of classification and adaptive segmen-
tation with the help of different statistical techniques. Malgouyres [13,14], Candès et
al. [15] have employed total variation minimization framework in a wavelet domain for
restoration of textured images. Casadei et al. [16] and Zhu et al. [17] have presented a
texture modeling with the help of statistical techniques.

Halidou et al. [18] proposed an approach for pedestrian detection by employing multi-block local binary pattern descriptors and region of interest (ROI). Here, three frame differencing and optical flow techniques are used to compute ROI. Neural network based classification approach and frame differencing technique are employed to detect moving objects in the indoor environments [19]. In this technique, robust classification is achieved using finite state automation. Caballero et al. [20] presented a technique for human detection using frame differencing and optical flow. This technique is proposed for an infrared camera mounted on a mobile robot. The non-pyramidal based Lucas-Kanade and the frame differencing techniques are employed to detect target objects by using thermal signatures [21]. A weighted mixture of Gaussians based adaptive background method is proposed for real time tracking [22]. However, this technique is not good for non-static backgrounds and sudden illumination changes. The W4 and histogram based frame differencing approaches are used to detect the moving objects [23]. Maddalena et al. [24] presented a self-organizing approach for background subtraction. In this approach, the background of the frame is learned with the help of labeling decision and neural network at neighboring pixels. Principal component analysis is used to projecting the high-dimensional data into a lower dimensional subspace for the background modeling [25]. This technique is suitable for global illumination variations, but slow moving objects in local illumination variation environment cannot be detected accurately. Sengar et al. [26] proposed two methods, viz., (i) improved three frame differencing and (ii) combined improved three frame differencing and background subtraction to accurately detect the slow/fast moving objects of varying number and size in the illumination variations and background clutter environments. Quaternion is used with modified optical flow technique to increase the accuracy of information of unclear pixels [27]. Schwarz et al. [28] employed the depth information obtained by a time of flight camera to compute the location of the foreground object. Reduced processing time and accurate results are obtained from the above techniques [27,28], but these are not suitable for real time environments. Sengar et al. [29] used the normalized self adaptive optical flow technique for moving object detection. Here, self adaptive window approach and Otsu's [30,31] method are used to select the target object area and to adapt the threshold value respectively. This method accurately detects the foreground, but not suitable for moving objects of small size.

## 3 Proposed method

The steps for the proposed motion segmentation technique are as follows:
**Algorithm 1:** Algorithm of Osher-Vese for structure-texture decomposition [32]

**Input**: Video frame $F$, No_of_iteration, Tuning parameter $\lambda > 0$, Step space $h > 0$

1. $U \leftarrow F$, function $f_1 = -\frac{F_x}{2\lambda|\nabla V_f|}$, function $f_2 = -\frac{F_y}{2\lambda|\nabla V_f|}$

2. **for** $t=1$ **to** $No\_of\_iteration$ **do**

**for** $i=2$ **to** $w-1$ **do**

**for** $j=2$ **to** $h-1$ **do**

$$K_1 \leftarrow \frac{1}{\sqrt{\left(\frac{U(i+1,j)}{h} - U(i,j)\right)^2 + \left(\frac{U(i,j+1)}{2h} - U(i,j-1)\right)^2}}$$

$$K_2 \leftarrow \frac{1}{\sqrt{\left(\frac{U(i,j)}{h} - U(i-1,j)\right)^2 + \left(\frac{U(i-1,j+1)}{2h} - U(i-1,j-1)\right)^2}}$$

$$K_3 \leftarrow \frac{1}{\sqrt{\left(\frac{U(i+1,j)}{2h} - U(i-1,j)\right)^2 + \left(\frac{U(i,j+1)}{h} - U(i,j)\right)^2}}$$

$$K_4 \leftarrow \frac{1}{\sqrt{\left(\frac{U(i+1,j-1)}{2h} - U(i-1,j-1)\right)^2 + \left(\frac{U(i,j)}{h} - U(i,j-1)\right)^2}}$$

$$U(i,j) \leftarrow \left( \frac{1}{1 + \frac{1}{2\lambda h^2}(K_1 + K_2 + K_3 + K_4)} \right) \times \left[ F(i,j) - \frac{f_1(i+1,j) - f_1(i-1,j)}{2h} - \frac{f_2(i,j+1) - f_2(i,j-1)}{2h} + \frac{1}{2\lambda h^2}(K_1 U(i+1,j) + K_2 U(i-1,j) + K_3 U(i,j+1) + K_4 U(i,j-1)) \right]$$

**end**

**end**

$$3. V = F - U$$

**end**

**Output:** Structure component ($U$) and texture/noise component ($V$)

* Extraction of frames from the video data.

* Conversion of video frame to gray-scale, if it is required.

* Separation of structure and texture/noise components.

* Application of three-frame differencing techniques on the structure components. Two difference frames are obtained.

* Computation of pixel-wise maximum value between estimated difference frames.

* Division of computed result of previous step in non-overlapping blocks.

* Calculation of intensity sum and intensity median of each block.

* Separation of background and foreground using threshold and block's median.

* Application of post processing steps to accurately detect the foreground objects.

The steps for the proposed technique is explained below:

## 3.1 Structure-texture image decomposition

A set of information in the form of structure, texture, and noise are contained by an image and it can be extracted using structure-texture image decomposition method. The inverse estimation problem i.e. structure-texture decomposition approach is mandatory for analyzing and understanding images. This method split a given video frame or image ($F$) into a structure (bounded variation) component $U$ and a texture and/or noise (oscillating) components $V$. i.e. $F = U + V$. Many models have been presented for image
<figure><img src="image_3.png" /><figcaption>Fig. 1: Schematic diagram of the proposed technique</figcaption></figure>

decomposition, in which one method is proposed by Osher and Vese [32]. This method has shown its efficiency by preserving discontinuities and removing oscillations in the image. The algorithmic steps for Osher-Vese [32] method are shown in Algorithm 1. Here in the algorithm, *w* and *h* denote the width and height of video frame.

<table><caption>Table 1: Description of test video dataset</caption><thead><tr><th>Description of video data</th><th>Frame size in pixels</th><th>Number of frames</th><th>Frame rate</th><th>Bit rate</th></tr></thead><tbody><tr><td>Video 1: In this dataset two men are moving in hall with briefcase in their hands. The color of the foreground and background are not much varied, but there is variation in illumination. (Color)</td><td>240×352</td><td>299</td><td>30</td><td>24</td></tr><tr><td>Video 2 In this dataset a man is walking slowly from one side of the frame to other side. The color of foreground is lighter than the background. (Color)</td><td>240×368</td><td>132</td><td>25</td><td>24</td></tr><tr><td>Video 3: In this dataset a lady is running. There are high amount of noise and illumination variation. (Color)</td><td>92×144</td><td>42</td><td>35</td><td>24</td></tr></tbody></table>

## 4 Implementation

The sequence of steps is presented below for the implementation of our technique:
## 4.1 Extraction of frames from video data and conversion to gray-scale

Frames from the input video data $D: Z \times Z \times T \rightarrow Z$ are extracted. In case, $D(x, y, t|x, y \in Z, t \in T) = N \in Z$. The $j^{th}$ frame can be represented as $F_j(x, y) = D(x, y, t = tj)$. Here in the next step, we focus on gray-scale frames in place of colored ones, due to the high complexity of three dimensional structures of color frames. Therefore, gray-scale frames can be obtained from colored one by employing [5]:

$$F(x, y) = 0.2126R(x, y) + 0.7152G(x, y) + 0.0722B(x, y) \quad (1)$$

Here R, G and B represent the red, green and blue channel components of each colored pixel, and scalar multiples represent the weights of these components.

## 4.2 Structure-Texture decomposition

The individual video frame extracted in the preceding step can have noise, and our target is to decrease the effect of noise for detection of accurate moving objects in the subsequent step. For that, structure and texture/noise decomposition technique is employed on individual frame with the help of Algorithm 1 and for the further processing, only the structure component (U) are being utilized.

## 4.3 Improved three-frame differencing

We employ the improved three frame differencing technique to find the pixel-wise difference of consecutive frames. It can be summarized in the following steps:

* Difference frame $Df_{t,t-1}(x, y)$ between frames $U_{t-1}(x, y)$ and $U_t(x, y)$ as well as second difference frame $Df_{t+1,t}(x, y)$ between frames $U_{t+1}(x, y)$ and $U_t(x, y)$ are estimated:

$$Df_{t,t-1}(x, y) = |U_{t-1}(x, y) - U_t(x, y)| \quad (2)$$

$$Df_{t+1,t}(x, y) = |U_{t+1}(x, y) - U_t(x, y)| \quad (3)$$

Here $U_{t-1}(x, y)$, $U_t(x, y)$, and $U_{t+1}(x, y)$ are the structure components of three successive video frames.

* Pixel-wise maximum intensity value between two difference frames is computed:

$$Df_t(x, y) = \max[Df_{t,t-1}(x, y), Df_{t+1,t}(x, y)] \quad (4)$$

## 4.4 Divide in non-overlapping blocks

Computed difference frame in previous step is partitioned into 8×8 size non-overlapping blocks.
<figure><img src="image_4.png" /><figcaption>Fig. 2: Results with Video 1 for the frames number (a) 72<sup>th</sup> (b) 191<sup>th</sup> (c) 222<sup>th</sup> (d) 265<sup>th</sup>; row wise, up to down: original frame, ground truth, Sukumaran [33], Fei [3], Yin [34], Sengar [5], and the proposed method.</figcaption></figure>

## 4.5 Detection of moving objects

This process can be summarized in the following steps:

* Intensity sum and median of each non-overlapping block are computed.

* Classification of background and foreground on the basis of block's median (Bm) and threshold (Th) i.e. if the value of block's intensity and block's sum are less than block's median and threshold respectively then it will be background (0), otherwise foreground (1).

$$FG_t(x, y) = \begin{cases} 0, & Df_t(x, y) < Bm \text{ and } sum(B_{tj}) < Th \\ 1, & \text{otherwise} \end{cases} \qquad (5)$$

Here $FG_t$, $sum(B_{tj})$ denote the foreground objects for the $t^{th}$ frame, sum of intensities of the $j^{th}$ block of the $t^{th}$ frame respectively.
<figure><img src="image_5.png" /><figcaption>Fig. 3: Results with Video 2 for the frames number (a) 4<sup>th</sup> (b) 34<sup>th</sup> (c) 81<sup>th</sup> (d) 119<sup>th</sup>; row wise, up to down: original frame, ground truth, Sukumaran [33], Fei [3], Yin [34], Sengar [5], and the proposed method.</figcaption></figure>

• Motion is detected by combining both the foreground and background pixels.

## 4.6 Post-processing

In morphology, closing is a cascaded combination of first dilation followed by erosion [35] and it is employed to fill the gap within the foreground object. Connected component labeling is employed to remove the isolated noisy blobs. Finally, morphological closing operations of square structuring element (SE) of size 3 × 3 (as shown in Eq. 6) followed by connected components labeling are employed to accurately segment the motion.

$$MV_t = (FG_t \oplus SE) \ominus SE \qquad (6)$$

The schematic diagram of our technique is shown by Fig. 1.
<figure><img src="image_6.png" /><figcaption>Fig. 4: Results with Video 3 for the frames number (a) 3<sup>th</sup> (b) 11<sup>th</sup> (c) 28<sup>th</sup> (d) 38<sup>th</sup>; row wise, up to down: original frame, ground truth, Sukumaran [33], Fei [3], Yin [34], Sengar [5], and the proposed method.</figcaption></figure>

# 5 Experimental results and analysis

We have done qualitative and quantitative evaluation to prove the efficiency of our technique.

## 5.1 Qualitative evaluation

To prove the effectiveness of our method, the proposed and existing techniques have been implemented and tested on standard benchmark video datasets namely: Video 1 (Hall monitoring) [36], Video 2 (Walk) [37], and Video 3 (Daria) [38]. The details of used video sequences are given in Table 1. We have iterated the structure-texture decomposition algorithm 100 times. The qualitative results of our algorithm as well as other tested methods on different video sequences are shown in Figs. 2-4. These
above mentioned figures display the original video frames, ground truth, and the motion segmentation results in each video frame by the five different techniques independently. For Video 1 we have displayed the experimental results on frame number 72, 191, 222, and 265. Similarly the related frame numbers are (4, 34, 81, 119), and (3, 11, 28, 38) for Video 2 and Video 3 respectively. The original frames and their ground truths are displayed by the first two rows of Figs. 2-4 respectively. Next five rows (from up to down) show the outcomes of Sukumaran [33], Fei [3], Yin [34], Sengar [5] and the Proposed technique respectively. It is proved from the aforementioned results that our approach in every case has considerably higher likeness with the ground truth as compared to existing approaches. All the existing approaches mis-classify most of the foreground pixels as background and vice-versa for all the tested video datasets. Hence, target objects are detected with holes and foreground pixels cannot be separated from the background. Furthermore, we have also measured and compared the performance of our technique based on quantitative evaluation in next section.

## 5.2 Quantitative evaluation

Quantitative evaluation and comparison with the help of accuracy and processing time are also done in our work. The experimental results of the proposed and existing techniques are compared with the ground truth (Figs. 2-4). Accuracy of the results can be computed using following equation:

$$count = \sum_{x=1}^{h} \sum_{y=1}^{w} G_{t}(x, y) (XOR) MV_{t}(x, y) \quad (7)$$

$$accuracy = 100 - \frac{count}{framesize} \times 100 \quad (8)$$

Where, the ground truth and obtained result of $t^{th}$ frame are represented by $G_t$ and $MV_t$ respectively. The number of mismatched pixels between ground truth and result are denoted by variable count. The results of quantitative evaluation are given in

<table><caption>Table 2: Average accuracy of existing and proposed schemes.</caption><thead><tr><th rowspan="2">Input video</th><th colspan="5">Scheme</th></tr><tr><th>Sukumaran [33]</th><th>Fei [3]</th><th>Yin [34]</th><th>Sengar [5]</th><th>Proposed</th></tr></thead><tbody><tr><td>Video 1</td><td>88.9</td><td>89.4</td><td>86.7</td><td>92.4</td><td>96.8</td></tr><tr><td>Video 2</td><td>93.2</td><td>92.9</td><td>92.5</td><td>89.9</td><td>96.6</td></tr><tr><td>Video 3</td><td>90.5</td><td>93.3</td><td>92.5</td><td>93.8</td><td>94.6</td></tr></tbody></table>

<table><caption>Table 3: Processing time of existing and proposed schemes.</caption><thead><tr><th rowspan="2">Input video</th><th colspan="5">Scheme</th></tr><tr><th>Sukumaran [33]</th><th>Fei [3]</th><th>Yin [34]</th><th>Sengar [5]</th><th>Proposed</th></tr></thead><tbody><tr><td>Video 1</td><td>1.01</td><td>0.90</td><td>0.87</td><td>0.95</td><td>0.83</td></tr><tr><td>Video 2</td><td>1.05</td><td>1.04</td><td>0.98</td><td>0.99</td><td>0.91</td></tr><tr><td>Video 3</td><td>0.79</td><td>0.75</td><td>0.73</td><td>0.80</td><td>0.65</td></tr></tbody></table>

Tables 2 and 3, these results are computed using different approaches over different datasets. The comparable results are also shown with the help of bar chart in Figs. 5 and 6 for accuracy and processing time respectively. It has been agreed from accuracy and processing time based quantitative evaluation that the proposed technique gives better results in comparison with other techniques.
<figure><img src="image_7.png" /><figcaption>Fig. 5: Comparison in the form of detection accuracy</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6: Comparison in the form of processing time</figcaption></figure>

## 6 Conclusion

Different techniques for motion segmentation have been analyzed and we came at conclusion that detection accuracy of these techniques is not suitable because of large amount of noise in the input video dataset. For that, we have proposed a simple and efficient approach for motion segmentation from complex noise scenes. With the help of proposed structure-texture decomposition and improved three frames differencing based scheme, we eliminate almost all the background noises without loss of foreground. Therefore, optimal results are obtained finally. This approach has improvements in comparison with other recent methods in the form of enhanced accuracy and decreased processing time. Experimental results and analyses prove that the proposed algorithm can effectively handle a slow/fast moving objects, camera shake problem and variation of object size with noisy datasets. In future attempt, our technique will be improved for visual object tracking in moving camera environment.
References

1. O. Elharrouss, D. Moujahid, and H. Tairi. Motion detection based on the combining of the background subtraction and the structure–texture decomposition. *Optik-International Journal for Light and Electron Optics*, 126(24):5992–5997, 2015.

2. S. S. Sengar and S. Mukhopadhyay. Detection of moving objects based on enhancement of optical flow. *Optik-International Journal for Light and Electron Optics*, 145:130–141, 2017.

3. M. Fei, J. Li, and H. Liu. Visual tracking based on improved foreground detection and perceptual hashing. *Neurocomputing*, 152:413–428, 2015.

4. S. S. Sengar and S. Mukhopadhyay. A novel method for moving object detection based on block based frame differencing. In *3rd International Conference on Recent Advances in Information Technology*, pages 462–472. IEEE, 2016.

5. S. S. Sengar and S. Mukhopadhyay. Motion detection using block based bi-directional optical flow method. *Journal of Visual Communication and Image Representation*, 49:89–103, 2017.

6. P. K. Sahoo, P. Kanungo, and S. Mishra. A fast valley-based segmentation for detection of slowly moving objects. *Signal, Image and Video Processing*, pages 1–8, 2018.

7. T. Bouwmans, A. Sobral, S. Javed, S. K. Jung, and E. H. Zahzah. Decomposition into low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a large-scale dataset. *Computer Science Review*, page., 2016.

8. S. S. Sengar and S. Mukhopadhyay. Moving object tracking using laplacian-dct based perceptual hash. In *International Conference on Wireless Communications, Signal Processing and Networking*, pages 2345–2349. IEEE, 2016.

9. B. Sandberg, T. Chan, and L. Vese. A level-set and gabor-based active contour algorithm for segmenting textured images. In *UCLA Department of Mathematics CAM report*. Citeseer, 2002.

10. S. Ono, T. Miyata, and I. Yamada. Cartoon-texture image decomposition using blockwise low-rank texture characterization. *IEEE Transactions on Image Processing*, 23(3):1128–1142, 2014.

11. J. Jeon, S. Cho, X. Tong, and S. Lee. Intrinsic image decomposition using structure-texture separation and surface normals. In *European Conference on Computer Vision*, pages 218–233. Springer, 2014.

12. W. M. Wells, W. E. L. Grimson, R. Kikinis, and F. A. Jolesz. Adaptive segmentation of mri data. *IEEE transactions on medical imaging*, 15(4):429–442, 1996.

13. F. Malgouyres. Mathematical analysis of a model which combines total variation and wavelet for image restoration. *Journal of information processes*, 2(1):1–10, 2002.

14. F. Malgouyres. Combining total variation and wavelet packet approaches for image deblurring. In *Variational and Level Set Methods in Computer Vision, 2001. Proceedings. IEEE Workshop on*, pages 57–64. IEEE, 2001.

15. E. J. Candès and F. Guo. New multiscale transforms, minimum total variation synthesis: Applications to edge-preserving image reconstruction. *Signal Processing*, 82(11):1519–1543, 2002.

16. S. Casadei, S. Mitter, and P. Perona. Boundary detection in piecewise homogeneous textured images. In *European Conference on Computer Vision*, pages 174–183. Springer, 1992.

17. S. C. Zhu, Y. Wu, and D. Mumford. Filters, random fields and maximum entropy (frame): Towards a unified theory for texture modeling. *International Journal of Computer Vision*, 27(2):107–126, 1998.

18. A. Halidou, X. You, M. Hamidine, R. A. Etoundi, and L. H. Diakite. Fast pedestrian detection based on region of interest and multi-block local binary pattern descriptors. *Computers & Electrical Engineering*, 40(8):375–389, 2014.
19. G. L. Foresti, C. Micheloni, and C. Piciarelli. Detecting moving people in video streams. *Pattern Recognition Letters*, 26(14):2232-2243, 2005.

20. A. F. Caballero, J. C. Castillo, J. M. Cantos, and R. M. Tomas. Optical flow or image subtraction in human detection from infrared camera on mobile robot. *Journal of Robotics and Autonomous Systems*, 58:1273-1281, 2010.

21. J. Y. Bouguet. Pyramidal implementation of the affine Lucas kanade feature tracker description of the algorithm. *Intel Corporation*, 5:1-10, 2001.

22. C. Stauffer and W. Grimson. Adaptive background mixture models for real-time tracking. In *International Conference On Computer Vision and Pattern Recognition*. IEEE, 1999.

23. S. S. Sengar and S. Mukhopadhyay. Moving object detection based on frame difference and w4. *Signal, Image and Video Processing*, 11(7):1357-1364, 2017.

24. L. Maddalena and A. Petrosino. The SOBS algorithm: what are the limits? In *Workshop on Computer Vision and Pattern Recognition*, pages 21–26. IEEE, 2012.

25. N. M. Oliver, B. Rosario, and A. P. Pentland. Bayesian computer vision system for modeling human interactions. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 22:831–843, 2000.

26. S. S. Sengar and S. Mukhopadhyay. Foreground detection via background subtraction and improved three-frame differencing. *Arabian Journal for Science and Engineering*, 42(8):3621-3633, 2017.

27. E. Chen, X. Xu, X. Yang, and W. Zhang. Quaternion based optical flow estimation for robust object tracking. *Journal of Digital Signal Processing*, 23:118–125, 2013.

28. L. A. Schwarz, A. Mkhitaryan, D. Mateus, and N. Navab. Human skeleton tracking from depth data using geodesic distances and optical flow. *Journal of Image and Vision Computing*, 30:217-226, 2012.

29. S. S. Sengar and S. Mukhopadhyay. Moving object area detection using normalized self adaptive optical flow. *Optik-International Journal for Light and Electron Optics*, 127(16):6258-6267, 2016.

30. D. Liu and J. Yu. Otsu method and k-means. In *9th International Conference on Hybrid Intelligent Systems*, pages 344–349. IEEE, 2009.

31. P. Liao, T. Chen, and P. Chung. A fast algorithm for level thresholding. *Journal of Information Science and Engineering*, 17:713-727, 2001.

32. A. V. Luminita and J. O. Stanley. Modeling textures with total variation minimization and oscillating patterns in image processing. *Journal of Scientific Computing*, 19(1-3):553-572, 2003.

33. A. N. Sukumaran, R. Sankararajan, and M. Swaminathan. Compressed sensing based fore-ground detection vector for object detection in wireless visual sensor networks. *AEU-International Journal of Electronics and Communications*, 72:216-224, 2017.

34. J. Yin, L. Liu, H. Li, and Q. Liu. The infrared moving object detection and security detection related algorithms based on w4 and frame difference. *Infrared Physics & Technology*, 77:302-315, 2016.

35. E. R. Dougherty and R. A. Lotufo. Hands-on morphological image processing. volume 71. Washington: SPIE Optical Engineering Press, 2003.

36. Database: Images & video clips (2). http://see.xidian.edu.cn/vips1/database_Video.html. Collected by the HDTV group, July, 2006.

37. vidme. https://vid.me/videodata. videodata, July, 2015.

38. Action Recognition. https://github.com/hueihan>Action_Recognition/tree/master/data/WIS/video/run.