# Using Sensor Network in Motion Detection Based on Deep Full Convolutional Network Model

Authors: Qichang Xu
Journal: Complexity
 (2021)

---

## Abstract

Aiming at the shortcomings of traditional moving target detection methods in complex scenes such as low detection accuracy and high complexity, and not considering the overall structure information of the video frame image, this paper proposes a moving-target detection based on sensor network. First, a low-power motion detection wireless sensor network node is designed to obtain motion detection information in real time. Secondly, the background of the video scene is quickly extracted by the time domain averaging method, and the video sequence and the background image are channel-merged to construct a deep full convolutional network model. Finally, the network model is used to learn the deep features of the video scene and output the pixel-level classification results to achieve moving target detection. This method not only can adapt to complex video scenes of different sizes but also has a simple background extraction method, which effectively improves the detection speed.


---

## Body

## 1. Introduction

Moving target detection refers to the research process of using computer vision methods to eliminate the irrelevant redundant information in the temporal and spatial representation spaces of the video, and to effectively extract the target that changes in the spatial position from it [1–4]. The wireless sensor network (WSN) functions by broadcasting a large number of sensor nodes in the interest area and rapidly forming a wireless network through self-organization. Using the additional information of these physical quantities, relying on the collaborative perception and computing capabilities of each node in the wireless sensor network, the location of the target object can be measured [5–7]. At present, the detection of moving targets using sensor networks has become a hot topic in the field of computer vision research.Target detection is an important task of wireless sensor networks [8–11], especially for military deployment. Zhou et al. [12] extracted the moving target in the scene by comparing the difference between two adjacent frames in the time series. Sengar et al. [13] improved the interframe difference method. First, the Canny algorithm is used to detect the edges of adjacent frames, and then the edge interpolation image is divided into small blocks, and the motion target is determined by counting the number of nonzero pixels in the small blocks. Zhong et al. [14] proposed a background modelling method based on the median method. Firstly, RGB images are transformed into HSV images. Secondly, the median value of HSV image is calculated to realize background modelling. This method can obtain better experimental results under the condition of normal illumination variation. But when the change of illumination is more drastic, the effect of HSV conversion space becomes bad. Therefore, the detection effect is poor. In addition to HSV transformation, the computation is increased, which leads to high time and space complexity. Gui et al. [15] proposed a background modelling method based on a single Gaussian model, which divides pixels into background or foreground by setting a specific threshold. Dou et al. [16] proposed a background modelling method of Gaussian mixture model (GMM). Yoo et al. [17] improved the background modelling method of the Gaussian mixture model, so that the number of Gaussian models in the algorithm can be calculated adaptively. Chen et al. [18] proposed a parameter-free kernel density estimation (KDE) method for moving target detection. Frutos-Valle et al. [19] proposed a moving target detection method based on principal component analysis, which can represent online video content while reducing data dimensions. Ilina et al. [20] proposed a moving target detection method based on sample consistency (SACON). Sun et al. [21] proposed a background modelling method (ViBe) based on pixel classification. The advantage of this method is that it does not need to assume any probability model, directly model a single pixel value, and determine the category of the new pixel based on the Euclidean distance. Yang et al. [22] combined the advantages of the SACON algorithm and the ViBe algorithm and improved them and proposed a pixel-based adaptive segmentation algorithm (PBAS). Haskins et al. [23] proposed the code book method. Liu et al. [24] proposed an adaptive moving target detection method based on K-means. Alpaslan et al. [25] applied LBP to background modelling and proposed a moving target detection method based on texture features. Dominguez et al. [26] improved the texture representation method, proposed a scale-invariant local ternary pattern (SILTP), and applied it to moving target detection. Novelli et al. [27] proposed a self-organizing neural network (SOBS) background modelling method. Gracewell et al. [28] improved the background modelling method based on self-organizing neural network, introduced spatial consistency into the background update stage, and proposed an improved self-organizing neural background modelling method (SC_SOBS). Shi et al. [29] proposed a scene-specific moving target detection method based on convolutional neural networks. Guo et al. [30] constructed a convolutional neural network model based on image blocks to achieve moving target detection based on a single pixel.In view of the shortcomings of the above algorithms, this chapter proposes a moving target detection algorithm based on a deep full convolutional network of sensor networks. First, a low-power sensor network node design is proposed. Secondly, the background of the video scene is quickly extracted by the time domain averaging method, and the video sequence and the background image are channel-merged to construct a deep full convolutional network model. Finally, the network model is used to learn the deep features of the video scene to distinguish the difference in details between the current frame image and the background image and output the pixel-level classification results to achieve moving target detection. Since the existence of deconvolution in the full convolutional network ensures that the output of the network is consistent with the input size, the overall structure information of the image is retained. When moving target detection, it not only effectively improves the detection accuracy, but also reduces time complexity of detection.


## 2. Basic Theory of Motion Detection

### 2.1. Image Features in Motion Detection

Image feature refers to the mathematical means of characterizing digital image processing and is the foundation of image processing technology. Generally speaking, as researchers deepen their understanding of images, image representation methods have also undergone an evolutionary process from shallow to deep [31–34]. Among them, the shallow representation of the image is mainly embodied in intuitive forms such as color features, geometric features, and shape features. The deep representation of the image is mainly driven by deep learning technology, and its potential visual representation features are extracted from different layers of the deep learning model.


#### 2.1.1. Color Characteristics

Color feature generally refers to the method of expressing color through mathematics; color information can be embodied and digitized. In the corresponding color space, a point in the three-dimensional space has a unique correspondence with a certain color.


#### 2.1.2. Geometric Features

In addition to color features, geometric features are also an effective means to characterize image visual features. Geometric features generally focus on special shapes such as edges and angles in the image.


#### 2.1.3. Texture Characteristics

Texture feature is a feature form commonly used in the shallow characterization of images in addition to color features and geometric features.


#### 2.1.4. Depth Feature Representation of the Image

In traditional machine learning algorithms, key issues such as which feature to choose and how many features to choose often have a great impact on the effect of the model. If the number of features is too small, the target object may not be accurately characterized, which indirectly leads to underfitting. Therefore, in order to solve the above-mentioned outstanding problems, more and more researchers now choose to use deep learning technology to automatically select the deep feature representation of the image. Among them, as one of the most popular deep learning models, convolutional neural networks have been widely used.


### 2.2. Postprocessing Methods in Motion Detection

#### 2.2.1. Image Filtering Processing

For the predicted image, the average filter is used to process the predicted image to obtain a relatively smooth segmented image and prevent the detected moving target area from having holes or discontinuities.


#### 2.2.2. Image Banalization

Image banalization is an important part of image processing. Through threshold segmentation of the original image, a binary image that can represent the original image information is obtained. In the moving target detection process, the difference image between the target image and the background image needs to obtain a binaries image to determine whether the current pixel belongs to the moving target or the background.


#### 2.2.3. Morphological Processing of Images

In the task of moving target detection, morphological processing is usually required for the detected binary image. Morphological processing can eliminate the isolated points in the binary image of moving target detection and fill the holes around the target to obtain a continuous and accurate target area to improve the accuracy of moving target detection. Common basic operations in morphological processing methods include corrosion, expansion, open operation, and close operation.


## 2.1. Image Features in Motion Detection

Image feature refers to the mathematical means of characterizing digital image processing and is the foundation of image processing technology. Generally speaking, as researchers deepen their understanding of images, image representation methods have also undergone an evolutionary process from shallow to deep [31–34]. Among them, the shallow representation of the image is mainly embodied in intuitive forms such as color features, geometric features, and shape features. The deep representation of the image is mainly driven by deep learning technology, and its potential visual representation features are extracted from different layers of the deep learning model.


### 2.1.1. Color Characteristics

Color feature generally refers to the method of expressing color through mathematics; color information can be embodied and digitized. In the corresponding color space, a point in the three-dimensional space has a unique correspondence with a certain color.


### 2.1.2. Geometric Features

In addition to color features, geometric features are also an effective means to characterize image visual features. Geometric features generally focus on special shapes such as edges and angles in the image.


### 2.1.3. Texture Characteristics

Texture feature is a feature form commonly used in the shallow characterization of images in addition to color features and geometric features.


### 2.1.4. Depth Feature Representation of the Image

In traditional machine learning algorithms, key issues such as which feature to choose and how many features to choose often have a great impact on the effect of the model. If the number of features is too small, the target object may not be accurately characterized, which indirectly leads to underfitting. Therefore, in order to solve the above-mentioned outstanding problems, more and more researchers now choose to use deep learning technology to automatically select the deep feature representation of the image. Among them, as one of the most popular deep learning models, convolutional neural networks have been widely used.


## 2.1.1. Color Characteristics

Color feature generally refers to the method of expressing color through mathematics; color information can be embodied and digitized. In the corresponding color space, a point in the three-dimensional space has a unique correspondence with a certain color.


## 2.1.2. Geometric Features

In addition to color features, geometric features are also an effective means to characterize image visual features. Geometric features generally focus on special shapes such as edges and angles in the image.


## 2.1.3. Texture Characteristics

Texture feature is a feature form commonly used in the shallow characterization of images in addition to color features and geometric features.


## 2.1.4. Depth Feature Representation of the Image

In traditional machine learning algorithms, key issues such as which feature to choose and how many features to choose often have a great impact on the effect of the model. If the number of features is too small, the target object may not be accurately characterized, which indirectly leads to underfitting. Therefore, in order to solve the above-mentioned outstanding problems, more and more researchers now choose to use deep learning technology to automatically select the deep feature representation of the image. Among them, as one of the most popular deep learning models, convolutional neural networks have been widely used.


## 2.2. Postprocessing Methods in Motion Detection

### 2.2.1. Image Filtering Processing

For the predicted image, the average filter is used to process the predicted image to obtain a relatively smooth segmented image and prevent the detected moving target area from having holes or discontinuities.


### 2.2.2. Image Banalization

Image banalization is an important part of image processing. Through threshold segmentation of the original image, a binary image that can represent the original image information is obtained. In the moving target detection process, the difference image between the target image and the background image needs to obtain a binaries image to determine whether the current pixel belongs to the moving target or the background.


### 2.2.3. Morphological Processing of Images

In the task of moving target detection, morphological processing is usually required for the detected binary image. Morphological processing can eliminate the isolated points in the binary image of moving target detection and fill the holes around the target to obtain a continuous and accurate target area to improve the accuracy of moving target detection. Common basic operations in morphological processing methods include corrosion, expansion, open operation, and close operation.


## 2.2.1. Image Filtering Processing

For the predicted image, the average filter is used to process the predicted image to obtain a relatively smooth segmented image and prevent the detected moving target area from having holes or discontinuities.


## 2.2.2. Image Banalization

Image banalization is an important part of image processing. Through threshold segmentation of the original image, a binary image that can represent the original image information is obtained. In the moving target detection process, the difference image between the target image and the background image needs to obtain a binaries image to determine whether the current pixel belongs to the moving target or the background.


## 2.2.3. Morphological Processing of Images

In the task of moving target detection, morphological processing is usually required for the detected binary image. Morphological processing can eliminate the isolated points in the binary image of moving target detection and fill the holes around the target to obtain a continuous and accurate target area to improve the accuracy of moving target detection. Common basic operations in morphological processing methods include corrosion, expansion, open operation, and close operation.


## 3. Deep Learning Motion Detection Algorithm Based on Sensor Network

### 3.1. Low-Power Sensor Network Node Design

Sensor networks integrate multiple technologies such as sensors, networks, and wireless communications and have become a hot field in the development of network technology and sensor technology in recent years [35–38].The design of the wireless sensor node is the core of motion detection. It mainly collects motion information through the PIR motion detector and sends it to the wireless communication chip CC1310 after proper preprocessing. The CC1310 wireless communication chip has a radio frequency control subsystem, through its radio frequency control subsystem. The collected movement information is sent out via the on-board antenna, and the main control system receives the movement information through the on-board antenna. The wireless sensor node is composed of sensor module, wireless communication chip module, printed circuit board (PCB) antenna, and power module. The hardware structure is shown in Figure1. Since the output of the sensor is an extremely small signal, it is necessary to amplify the signal and filter out noise through amplification and filtering operations in order to obtain a suitable output signal and minimize false trigger events. The amplified analogy output signal is converted into a digital signal by a window comparator, which is used as an interrupt of the wireless communication chip and wakes up the CC1310 if necessary to achieve the purpose of energy saving.Figure 1
Structure diagram of sensor network.


#### 3.1.1. Wireless Communication Chip

The wireless communication chip is the core of the entire system. In order to save power consumption, the sensor can be processed by a dedicated ultra-low-power autonomous microcontroller unit (MCU) in the chip with ultra-low-power consumption. The MCU can be configured to process analogy and digital sensors, so the main MCU can maximize sleep time.


#### 3.1.2. PCB Antenna

The antenna is a space concept, and the performance of the PCB antenna is not as good as the independent antenna. However, considering the miniaturization and integrated design of the node, the PCB antenna is used in this design.


#### 3.1.3. PIR Motion Detection Module

The PIR motion sensor includes two or more components, and the output voltage of these components is proportional to the amount of incident infrared radiation.


#### 3.1.4. Data Preprocessing Module

In this design, it is necessary to amplify and filter the signal at the output of the PIR sensor, so that the amplitude of the signal entering the subsequent stages of the signal chain is sufficient to provide useful information. The filter circuit composed of the first stage and the second stage realizes a fourth-order band-pass filter with simple poles, and each circuit can achieve the same second-order band-pass filter characteristics.The first stage of the filter acts as a noninverting gain stage, which provides a high impedance load for the sensor to keep its bias point constant. Formula (1) is the first stage gain, formula (2) is the second stage gain, and formula (3) is the total loop gain.(1)AI=1+0.35⋅B1B2,(2)AII=1+0.5⋅B3B4,(3)A=0.17⋅AI+0.83⋅AII=0.17⋅1+0.35⋅B1B2+0.83⋅1+0.5⋅B3B4.


#### 3.1.5. Power Module

This design uses CR2032 lithium-ion button battery as the power source. The CR2032 button battery was selected as the power source because this type of battery is universally applicable, especially in small form factor systems (such as sensor terminal nodes).


### 3.2. Deep Full Convolutional Network Algorithm

This paper proposes a Deep Fully Convolutional Networks (DFCN) model for complex scene videos. First, the time domain averaging method is used to quickly extract the background image of the video, and then the video sequence image, the background image, and the ground truth image of the real moving target detection result are scaled to obtain the image sequence with the same size. After that, the original video sequence image and its corresponding background image are channel merged, part of the video sequence is selected as the training sample, and the corresponding ground truth is used as the sample set label to train the DFCN model. Finally, the trained model is used to detect moving targets in the video frames that have not participated in the training in the trained scene and verify the moving target detection effect of the trained model in the untrained scene.


#### 3.2.1. The Overall Framework of Moving Target Detection in Adaptive Scenes

In reality, the size of surveillance video is generally different. In order to enable the DFCN model proposed in this paper to learn from different video scenes, and to use the trained model to detect moving targets in different scenes, a bilinear interpolation algorithm is used to scale the video frame images to unify the network input size to realize the moving target detection of adaptive scene.The output of the DFCN model proposed in this paper is a two-dimensional matrix with the same size as the input image. It can be regarded as a semantic segmentation image. Each pixel value represents the probability that the corresponding pixel belongs to the background or foreground, and its value range is [0, 1]. For the semantic segmentation image output by the DFCN model, relevant image postprocessing operations are performed on it, including image mean filtering and image threshold, and finally the moving target detection result is obtained. The overall framework of adaptive scene moving target detection is shown in Figure2, which mainly includes five parts: video background extraction, image size scaling, channel merging, DFCN model, and image postprocessing.Figure 2
The overall framework of moving target detection for adaptive scenes.The DFCN model uses an encoder-decoder structure. The encoder part uses a convolutional neural network, whose purpose is to extract the deep features of the image through a series of convolution and pooling. For the decoder part, that is, the deconvolution network, it uses convolution and uppooling for upsampling and uses a skip structure to reconstruct the original image information. The configuration is shown in Table1.Table 1
The configuration of the method.
LayerKernel sizeChannel1(3, 3)102(3, 3)203(3, 3)404(3, 3)805(3, 3)1606(4, 4)3207(4, 4)6408(4, 4)12809(4, 4)64010(4, 4)32011(3, 3)16012(3, 3)8013(3, 3)4014(3, 3)2015(3, 3)10Cross entropy is used to measure the error of the network model. The mathematics of the loss function used in the expression is,(4)loss=1M∑iIi⋅lnOi+1−Iiln1−Oi.


#### 3.2.2. Dropout

In deep learning models, overfitting usually occurs. Overfitting means that the network model learns the data so thoroughly that it also learns the characteristics of the noisy data. This will cause the model to perform well on the validation set but poorly on the test set, making a well-trained deep network. The model cannot be applied well to untrained data. In order to avoid overfitting of the network model, methods such as recleaning the data, increasing the amount of data training, regularization, K-fold cross-validation, and dropout are usually adopted.Dropout, as a method to avoid overfitting, refers to randomly discarding some hidden layer neurons with a certain probability in the network model and keeping the input and output neurons unchanged. The dropout structure makes the fully connected network sparse to a certain extent, effectively reduces the synergy between different features, and improves the robustness of the neural network. Its structure is shown in Figure3. In the DFCN model, two dropout layers are set behind the network model to prevent the network from overfitting and improve the generalization ability of the network model. The rate of dropout is 0.5. This is an empirical value.Figure 3
Dropout operation.


### 3.3. Semantic Segmentation Image Postprocessing Method

The prediction result of the DFCN model is an image with the same size as the original input image, and the value range is [0, 1]. Each pixel value in the image represents the probability that the pixel belongs to the foreground or the background. In order to obtain a binary image of the moving target detection result consistent with the size of the original video frame image, and to further optimize the network output, the semantic segmentation image output by the DFCN model is subjected to the following postprocessing operations:(1)
Mean filter: filter out the random noise in the prediction result to ensure the continuity of the detected moving target.(2)
Threshold for the first time: the threshold Thr1 is used to obtain a binary image for moving target detection.(3)
Threshold for the second time: the new interpolation pixel in the bilinear interpolation algorithm is set by Thr2.(4)
Threshold for the third time: the preliminary segmentation results contain a lot of noise points, so we need to further determine which pixels actually contain moving objects to eliminate the false sun points of pixels without moving objects.In order to determine whether a pixel contains a moving object, this paper introduces the Q1 feature of the number of members adjacent to the cluster centre, using this feature to construct a simple histogram of the pixel. We use the histogram similarity metric announcement to determine the similaritys of two simple histograms of pixels:(5)s=1M∑iminQ2−Q1.Set a thresholdα2. If s>α2, the pixel has no obvious change, and the pixel is marked as background. If s ≤ α2, it means that the pixel has changed significantly. The standard deviation feature sd of the pixel is extracted by,(6)μ=1M∑iIi,(7)sd=1M∑iIi−μ21/2,where Ii is the grey value of the i-th pixel, and M is the number of pixels.Then, after calculating the similarity degree of the simple histogram of formula (5), it can be determined that a large number of pixels are marked as background without significant changes, without participating in subsequent calculations. After extracting the standard deviation sd feature of the pixel, this paper combines the standard deviation feature sd of the pixel, the number of previous scenic spots in the pixel Q2 feature, and the previously calculated simple histogram similarity s to propose a weighted similarity measure formula to distinguish the two happening:(8)Dis=1sαsd2−sd1+1−αQ2−Q1.Among them,α is the weight coefficient, and the value range is any decimal number from zero to one. Also, set a threshold Thr3, and Dis > Thr3 means that the pixel belongs to the situation that contains moving objects and should be marked as foreground. At the same time, the value of the pixel point mask matrix marked as the previous scenic spot inside the pixel is retained. If Dis ≤ Thr3, it means that the pixel belongs to the situation where the background model needs to be updated, and the pixel is marked to be updated, and the mask matrix belonging to the pixel is reset to zero.


## 3.1. Low-Power Sensor Network Node Design

Sensor networks integrate multiple technologies such as sensors, networks, and wireless communications and have become a hot field in the development of network technology and sensor technology in recent years [35–38].The design of the wireless sensor node is the core of motion detection. It mainly collects motion information through the PIR motion detector and sends it to the wireless communication chip CC1310 after proper preprocessing. The CC1310 wireless communication chip has a radio frequency control subsystem, through its radio frequency control subsystem. The collected movement information is sent out via the on-board antenna, and the main control system receives the movement information through the on-board antenna. The wireless sensor node is composed of sensor module, wireless communication chip module, printed circuit board (PCB) antenna, and power module. The hardware structure is shown in Figure1. Since the output of the sensor is an extremely small signal, it is necessary to amplify the signal and filter out noise through amplification and filtering operations in order to obtain a suitable output signal and minimize false trigger events. The amplified analogy output signal is converted into a digital signal by a window comparator, which is used as an interrupt of the wireless communication chip and wakes up the CC1310 if necessary to achieve the purpose of energy saving.Figure 1
Structure diagram of sensor network.


### 3.1.1. Wireless Communication Chip

The wireless communication chip is the core of the entire system. In order to save power consumption, the sensor can be processed by a dedicated ultra-low-power autonomous microcontroller unit (MCU) in the chip with ultra-low-power consumption. The MCU can be configured to process analogy and digital sensors, so the main MCU can maximize sleep time.


### 3.1.2. PCB Antenna

The antenna is a space concept, and the performance of the PCB antenna is not as good as the independent antenna. However, considering the miniaturization and integrated design of the node, the PCB antenna is used in this design.


### 3.1.3. PIR Motion Detection Module

The PIR motion sensor includes two or more components, and the output voltage of these components is proportional to the amount of incident infrared radiation.


### 3.1.4. Data Preprocessing Module

In this design, it is necessary to amplify and filter the signal at the output of the PIR sensor, so that the amplitude of the signal entering the subsequent stages of the signal chain is sufficient to provide useful information. The filter circuit composed of the first stage and the second stage realizes a fourth-order band-pass filter with simple poles, and each circuit can achieve the same second-order band-pass filter characteristics.The first stage of the filter acts as a noninverting gain stage, which provides a high impedance load for the sensor to keep its bias point constant. Formula (1) is the first stage gain, formula (2) is the second stage gain, and formula (3) is the total loop gain.(1)AI=1+0.35⋅B1B2,(2)AII=1+0.5⋅B3B4,(3)A=0.17⋅AI+0.83⋅AII=0.17⋅1+0.35⋅B1B2+0.83⋅1+0.5⋅B3B4.


### 3.1.5. Power Module

This design uses CR2032 lithium-ion button battery as the power source. The CR2032 button battery was selected as the power source because this type of battery is universally applicable, especially in small form factor systems (such as sensor terminal nodes).


## 3.1.1. Wireless Communication Chip

The wireless communication chip is the core of the entire system. In order to save power consumption, the sensor can be processed by a dedicated ultra-low-power autonomous microcontroller unit (MCU) in the chip with ultra-low-power consumption. The MCU can be configured to process analogy and digital sensors, so the main MCU can maximize sleep time.


## 3.1.2. PCB Antenna

The antenna is a space concept, and the performance of the PCB antenna is not as good as the independent antenna. However, considering the miniaturization and integrated design of the node, the PCB antenna is used in this design.


## 3.1.3. PIR Motion Detection Module

The PIR motion sensor includes two or more components, and the output voltage of these components is proportional to the amount of incident infrared radiation.


## 3.1.4. Data Preprocessing Module

In this design, it is necessary to amplify and filter the signal at the output of the PIR sensor, so that the amplitude of the signal entering the subsequent stages of the signal chain is sufficient to provide useful information. The filter circuit composed of the first stage and the second stage realizes a fourth-order band-pass filter with simple poles, and each circuit can achieve the same second-order band-pass filter characteristics.The first stage of the filter acts as a noninverting gain stage, which provides a high impedance load for the sensor to keep its bias point constant. Formula (1) is the first stage gain, formula (2) is the second stage gain, and formula (3) is the total loop gain.(1)AI=1+0.35⋅B1B2,(2)AII=1+0.5⋅B3B4,(3)A=0.17⋅AI+0.83⋅AII=0.17⋅1+0.35⋅B1B2+0.83⋅1+0.5⋅B3B4.


## 3.1.5. Power Module

This design uses CR2032 lithium-ion button battery as the power source. The CR2032 button battery was selected as the power source because this type of battery is universally applicable, especially in small form factor systems (such as sensor terminal nodes).


## 3.2. Deep Full Convolutional Network Algorithm

This paper proposes a Deep Fully Convolutional Networks (DFCN) model for complex scene videos. First, the time domain averaging method is used to quickly extract the background image of the video, and then the video sequence image, the background image, and the ground truth image of the real moving target detection result are scaled to obtain the image sequence with the same size. After that, the original video sequence image and its corresponding background image are channel merged, part of the video sequence is selected as the training sample, and the corresponding ground truth is used as the sample set label to train the DFCN model. Finally, the trained model is used to detect moving targets in the video frames that have not participated in the training in the trained scene and verify the moving target detection effect of the trained model in the untrained scene.


### 3.2.1. The Overall Framework of Moving Target Detection in Adaptive Scenes

In reality, the size of surveillance video is generally different. In order to enable the DFCN model proposed in this paper to learn from different video scenes, and to use the trained model to detect moving targets in different scenes, a bilinear interpolation algorithm is used to scale the video frame images to unify the network input size to realize the moving target detection of adaptive scene.The output of the DFCN model proposed in this paper is a two-dimensional matrix with the same size as the input image. It can be regarded as a semantic segmentation image. Each pixel value represents the probability that the corresponding pixel belongs to the background or foreground, and its value range is [0, 1]. For the semantic segmentation image output by the DFCN model, relevant image postprocessing operations are performed on it, including image mean filtering and image threshold, and finally the moving target detection result is obtained. The overall framework of adaptive scene moving target detection is shown in Figure2, which mainly includes five parts: video background extraction, image size scaling, channel merging, DFCN model, and image postprocessing.Figure 2
The overall framework of moving target detection for adaptive scenes.The DFCN model uses an encoder-decoder structure. The encoder part uses a convolutional neural network, whose purpose is to extract the deep features of the image through a series of convolution and pooling. For the decoder part, that is, the deconvolution network, it uses convolution and uppooling for upsampling and uses a skip structure to reconstruct the original image information. The configuration is shown in Table1.Table 1
The configuration of the method.
LayerKernel sizeChannel1(3, 3)102(3, 3)203(3, 3)404(3, 3)805(3, 3)1606(4, 4)3207(4, 4)6408(4, 4)12809(4, 4)64010(4, 4)32011(3, 3)16012(3, 3)8013(3, 3)4014(3, 3)2015(3, 3)10Cross entropy is used to measure the error of the network model. The mathematics of the loss function used in the expression is,(4)loss=1M∑iIi⋅lnOi+1−Iiln1−Oi.


### 3.2.2. Dropout

In deep learning models, overfitting usually occurs. Overfitting means that the network model learns the data so thoroughly that it also learns the characteristics of the noisy data. This will cause the model to perform well on the validation set but poorly on the test set, making a well-trained deep network. The model cannot be applied well to untrained data. In order to avoid overfitting of the network model, methods such as recleaning the data, increasing the amount of data training, regularization, K-fold cross-validation, and dropout are usually adopted.Dropout, as a method to avoid overfitting, refers to randomly discarding some hidden layer neurons with a certain probability in the network model and keeping the input and output neurons unchanged. The dropout structure makes the fully connected network sparse to a certain extent, effectively reduces the synergy between different features, and improves the robustness of the neural network. Its structure is shown in Figure3. In the DFCN model, two dropout layers are set behind the network model to prevent the network from overfitting and improve the generalization ability of the network model. The rate of dropout is 0.5. This is an empirical value.Figure 3
Dropout operation.


## 3.2.1. The Overall Framework of Moving Target Detection in Adaptive Scenes

In reality, the size of surveillance video is generally different. In order to enable the DFCN model proposed in this paper to learn from different video scenes, and to use the trained model to detect moving targets in different scenes, a bilinear interpolation algorithm is used to scale the video frame images to unify the network input size to realize the moving target detection of adaptive scene.The output of the DFCN model proposed in this paper is a two-dimensional matrix with the same size as the input image. It can be regarded as a semantic segmentation image. Each pixel value represents the probability that the corresponding pixel belongs to the background or foreground, and its value range is [0, 1]. For the semantic segmentation image output by the DFCN model, relevant image postprocessing operations are performed on it, including image mean filtering and image threshold, and finally the moving target detection result is obtained. The overall framework of adaptive scene moving target detection is shown in Figure2, which mainly includes five parts: video background extraction, image size scaling, channel merging, DFCN model, and image postprocessing.Figure 2
The overall framework of moving target detection for adaptive scenes.The DFCN model uses an encoder-decoder structure. The encoder part uses a convolutional neural network, whose purpose is to extract the deep features of the image through a series of convolution and pooling. For the decoder part, that is, the deconvolution network, it uses convolution and uppooling for upsampling and uses a skip structure to reconstruct the original image information. The configuration is shown in Table1.Table 1
The configuration of the method.
LayerKernel sizeChannel1(3, 3)102(3, 3)203(3, 3)404(3, 3)805(3, 3)1606(4, 4)3207(4, 4)6408(4, 4)12809(4, 4)64010(4, 4)32011(3, 3)16012(3, 3)8013(3, 3)4014(3, 3)2015(3, 3)10Cross entropy is used to measure the error of the network model. The mathematics of the loss function used in the expression is,(4)loss=1M∑iIi⋅lnOi+1−Iiln1−Oi.


## 3.2.2. Dropout

In deep learning models, overfitting usually occurs. Overfitting means that the network model learns the data so thoroughly that it also learns the characteristics of the noisy data. This will cause the model to perform well on the validation set but poorly on the test set, making a well-trained deep network. The model cannot be applied well to untrained data. In order to avoid overfitting of the network model, methods such as recleaning the data, increasing the amount of data training, regularization, K-fold cross-validation, and dropout are usually adopted.Dropout, as a method to avoid overfitting, refers to randomly discarding some hidden layer neurons with a certain probability in the network model and keeping the input and output neurons unchanged. The dropout structure makes the fully connected network sparse to a certain extent, effectively reduces the synergy between different features, and improves the robustness of the neural network. Its structure is shown in Figure3. In the DFCN model, two dropout layers are set behind the network model to prevent the network from overfitting and improve the generalization ability of the network model. The rate of dropout is 0.5. This is an empirical value.Figure 3
Dropout operation.


## 3.3. Semantic Segmentation Image Postprocessing Method

The prediction result of the DFCN model is an image with the same size as the original input image, and the value range is [0, 1]. Each pixel value in the image represents the probability that the pixel belongs to the foreground or the background. In order to obtain a binary image of the moving target detection result consistent with the size of the original video frame image, and to further optimize the network output, the semantic segmentation image output by the DFCN model is subjected to the following postprocessing operations:(1)
Mean filter: filter out the random noise in the prediction result to ensure the continuity of the detected moving target.(2)
Threshold for the first time: the threshold Thr1 is used to obtain a binary image for moving target detection.(3)
Threshold for the second time: the new interpolation pixel in the bilinear interpolation algorithm is set by Thr2.(4)
Threshold for the third time: the preliminary segmentation results contain a lot of noise points, so we need to further determine which pixels actually contain moving objects to eliminate the false sun points of pixels without moving objects.In order to determine whether a pixel contains a moving object, this paper introduces the Q1 feature of the number of members adjacent to the cluster centre, using this feature to construct a simple histogram of the pixel. We use the histogram similarity metric announcement to determine the similaritys of two simple histograms of pixels:(5)s=1M∑iminQ2−Q1.Set a thresholdα2. If s>α2, the pixel has no obvious change, and the pixel is marked as background. If s ≤ α2, it means that the pixel has changed significantly. The standard deviation feature sd of the pixel is extracted by,(6)μ=1M∑iIi,(7)sd=1M∑iIi−μ21/2,where Ii is the grey value of the i-th pixel, and M is the number of pixels.Then, after calculating the similarity degree of the simple histogram of formula (5), it can be determined that a large number of pixels are marked as background without significant changes, without participating in subsequent calculations. After extracting the standard deviation sd feature of the pixel, this paper combines the standard deviation feature sd of the pixel, the number of previous scenic spots in the pixel Q2 feature, and the previously calculated simple histogram similarity s to propose a weighted similarity measure formula to distinguish the two happening:(8)Dis=1sαsd2−sd1+1−αQ2−Q1.Among them,α is the weight coefficient, and the value range is any decimal number from zero to one. Also, set a threshold Thr3, and Dis > Thr3 means that the pixel belongs to the situation that contains moving objects and should be marked as foreground. At the same time, the value of the pixel point mask matrix marked as the previous scenic spot inside the pixel is retained. If Dis ≤ Thr3, it means that the pixel belongs to the situation where the background model needs to be updated, and the pixel is marked to be updated, and the mask matrix belonging to the pixel is reset to zero.


## 4. Results and Discussion

### 4.1. Data Set

The performance of the moving object detection algorithm proposed in this chapter is evaluated on the CDnet dataset. Considering the actual memory limitation, we used part of the CDnet data for experiments. The detailed experimental data information is shown in Table2.Table 2
Videos and detailed information used in the experiment.
Scene categoryScene nameNumber of video framesVideo sequence of interestSizeBaselinePedestrians1099299-1011360×240PETS20061200299-1011720×576Highway1899199-999360×240Bad weatherBlizzard32121011-2100720×480Skating1120100-500720×576Snowfall1313120-1100540×360Wet snow1287120-1100352×240Camera jitterSide walk1150100-1000720×480Traffic2198500-1500720×576Badminton1654500-1500540×360Boulevard1278120-1100352×240Dynamic backgroundBoats30001000-2000720×480Canoe1200120-1100720×576Overpass31001000-2000540×360Fall1200120-1100352×240


### 4.2. Parameter Selection

The parameter combination that performs best in a particular video sequence may not perform well in other video sequences. In order to make the algorithm get better performance, the experiment part has been carried on the relevant discussion of parameter setting. The algorithm in this paper has 6 parameters to be set; they are as follows:α is the weight coefficient used in formula (8), with a value range of 0 to 1. Symbol α determines the standard deviation feature of the superpixel, the absolute difference between the number feature of the superpixel and the background model, and the influence proportion in formula (8), which affects the judgment of whether the superpixel contains a moving object.Thr1 is a threshold. After calculating the absolute difference between the pixel points and the cluster centre one by one, these differences need to be compared with the threshold Thr1. If the difference is smaller than the threshold Thr1, it is marked as a background point, and all differences are greater than threshold Thr1 marked as the former scenic spot. The smaller the threshold Thr1, the more sensitive the algorithm, and the easier it is to detect moving objects, but at the same time there will be more noise.Thr2 is a threshold value that calculates the similaritys of the simple histogram by using the Q feature of the number of neighbouring pixel members of the cluster centre. If the degree of similarity s is greater than the threshold Thr2, it is marked as a background; otherwise, it is necessary to further determine the properties of the superpixel. Generally speaking, the larger the threshold Thr2, the better the performance of the algorithm in the experiment, but the computing time will also increase significantly.Thr3 is a threshold, and the weighted similarity measure Dis is calculated by formula (8). If the similarity measure Dis is greater than the threshold Thr3, the superpixel is marked as the foreground; otherwise, it is marked to be updated. Generally speaking, the smaller the threshold Thr3 is, the more sensitive the algorithm is, but if the threshold is too small, it will affect the background update part of the algorithm.K is the number of clustering centres. The larger the K, the more complex the model, and the higher the accuracy, but the algorithm will take more time. Too small K will lead to too many outliers, and it is impossible to construct a scientific background model.Taking F-score as an example, Figure4 shows the experimental results of the F-score value when the parameters α, Thr2, Thr3, and K are set to different values, and other parameters remain unchanged. It can be seen that, in addition to the parameter Thr2, if the values of the parameters α, Thr3, and K change within a certain interval, the F-score value will be higher, and the algorithm performance will be better, and when α, Thr3, and K are set to 0.75, 95, and 30, you can get the best results of the selected parameter experiment. Because the data set is large, it is currently impossible to conduct more detailed experiments, such as considering the simultaneous change of multiple parameters to verify whether there is an impact between the parameters, so, although there may be better parameter choices, the algorithm in this chapter still uses α, Thr3, and k set to 0.75, 95, and 30, respectively, and the other parameters are the same.Figure 4
Experimental results of performance index F-score whenα, Thr1, Thr2, Thr3, and K have different values. (a) F-score value of α and Thr2. (b) F-score value of K and Thr3.
(a)(b)Figure5 shows the F-score value and the running time of the experimental results when the parameter Thr2 is set to different values. The results show that when the Thr2 value gradually increases, the F-score value and running time will gradually increase, but the F-score value will increase. The speed gradually slows down or even falls back briefly, while the running time growth speed gradually increases, and the growth speed increases sharply in the range of 0.8 to 0.9. Therefore, considering the F-score value and running time, the algorithm in this chapter is used in all video sequences. Set the parameters Thr2 to 0.8.Figure 5
The experimental results of performance index F-score and running time when the value of Thr2 is different.


### 4.3. Performance Analysis

FCN has achieved great success in image segmentation. In order to verify whether FCN can be used in the field of moving target detection, experiments and analyses are carried out from the three aspects of the feasibility, reliability, and robustness of the DFCN model, and the DFCN method is compared with mainstream moving target detection algorithms.


#### 4.3.1. Algorithm Feasibility Experiment and Analysis

In order to test whether the DFCN model converges, the DFCN model is supervised and trained on the training set for 1000 iterations, and the error of the model on the training set and the test set is recorded. The error curve of the training set and the test set is shown in Figure6.Figure 6
Error curves of the training set, validation set, and test set of the DFCN model.It can be seen from Figure6 that the training set error gradually decreases with the number of iterations and stabilizes after 1000 iterations, and the error is basically zero. The error of the test set decreases with the increase of the number of iterations, but the phenomenon of oscillation appears.This is because the scene where the test set is located is not involved in training at all, and there are many similar pixels between adjacent frames of the scene involved in training. Therefore, for the scenes that have participated in the training, it can be considered that the DFCN model has basically converged, but the model still has a certain error in the scenes that have not participated in the training. In summary, the DFCN method proposed in this paper is feasible.


#### 4.3.2. Algorithm Reliability Experiment and Analysis

In order to verify the reliability of the DFCN method proposed in this chapter, the trained network model was tested on a total of 29 scenarios in 7 categories in the data set, and the average value of each evaluation index of the 7 scenarios was recorded with objective evaluation indicators.The average evaluation indicators of the DFCN method in various scenarios on the CDnet dataset are shown in Figure7. It can be seen from Figure 7 that the detection accuracy of the DFCN method in various complex scenarios in the CDnet data set is particularly high, both R and P are higher than 96%, and the comprehensive performance index of the DFCN method in the above complex scenarios is F-score, all are above 96%, and the FPR and FNR are extremely small.Figure 7
The average evaluation index results of the DFCN model in various scenarios. (a) Difference (R) S. (P) and F-score with number of iteration. (b) Difference P-score, FPR, and FNR with number of iterations.
(a)(b)In order to further illustrate the superiority of the DFCN method proposed in this paper over the existing methods, several representative moving target detection algorithms are selected for comparison, including GMM, PBAS, KDE, SACON, SILTP, and SC_SOBS. Figure8 shows the comparison result of the average comprehensive index F-score of different moving target detection algorithms in the 7 types of scenes in the CDnet dataset.Figure 8
F-score evaluation index results of DFCN model and other algorithms in various scenarios.It can be seen from Figure8 that the comprehensive performance index F-score value of the DFCN method in various complex scenarios on CDnet is far superior to other methods. Compared with the latest method, in baseline, bad weather, dynamic background, shadow, camera jitter, and thermal and night videos category scenarios, the average comprehensive performance F-score value has increased by 2.3%, 19.5%, 11.7%, and 6.7%. Therefore, the DFCN method proposed in this paper has been greatly improved.In order to verify whether the input data of different colour spaces affect the detection results of moving targets, the RGB colour space and the HSV colour space are, respectively, selected for experiments, and the average F-score value in various scenarios is tested. The comparison results are shown in Figure9.Figure 9
F-score value comparison of input data using different colour spaces.It can be seen from Figure9 that, in all 7 scene categories in the CDnet dataset, when the input data uses the RGB colour space, the average F-score value of the 4 scene categories is higher than that of the input data using the HSV space. The average F-score value of the three scene categories is lower than the input data using HSV space, and the F-score value is not much different. Therefore, in the DFCN method proposed in this paper, the selection of the colour space of the input data has little effect on the result of moving target detection.


#### 4.3.3. Algorithm Robustness Experiment and Analysis

In order to verify the robustness of the DFCN method proposed in this chapter, the trained DFCN model will be tested in other scenarios that are not involved in the training, and evaluation indicators will be calculated. In the experiment, 100% data of 28 videos except the highway video from the 29 videos selected from the CDnet dataset are used for training, and the trained DFCN model is used to test the highway videos that are not involved in the training. The model is in the highway video. The evaluation indicators of the following test results are shown in Table3.Table 3
Evaluation indicators of the DFCN model in the Highway scenario without training.
ScenarioRPSF-scoreFPRFNRP-scoreHighway0.9120.8120.9990.9360.0010.0870.712It can be seen from Table3 that the detection result of the DFCN method in the highway scene without training is not ideal. Among them, the F-score value is 0.936. Moreover, the FNR and P-score in the detection are extremely high, indicating that the DFCN model has a large number of missed detections on untrained highway videos. Experiments have proved that the DFCN method proposed in this paper can get extremely excellent results in the scenes that have participated in the training, but the detection accuracy of the model is not high in the scenes that have not participated in the training. Therefore, the robustness of the DFCN method proposed in this paper still needs to be further improved.


## 4.1. Data Set

The performance of the moving object detection algorithm proposed in this chapter is evaluated on the CDnet dataset. Considering the actual memory limitation, we used part of the CDnet data for experiments. The detailed experimental data information is shown in Table2.Table 2
Videos and detailed information used in the experiment.
Scene categoryScene nameNumber of video framesVideo sequence of interestSizeBaselinePedestrians1099299-1011360×240PETS20061200299-1011720×576Highway1899199-999360×240Bad weatherBlizzard32121011-2100720×480Skating1120100-500720×576Snowfall1313120-1100540×360Wet snow1287120-1100352×240Camera jitterSide walk1150100-1000720×480Traffic2198500-1500720×576Badminton1654500-1500540×360Boulevard1278120-1100352×240Dynamic backgroundBoats30001000-2000720×480Canoe1200120-1100720×576Overpass31001000-2000540×360Fall1200120-1100352×240


## 4.2. Parameter Selection

The parameter combination that performs best in a particular video sequence may not perform well in other video sequences. In order to make the algorithm get better performance, the experiment part has been carried on the relevant discussion of parameter setting. The algorithm in this paper has 6 parameters to be set; they are as follows:α is the weight coefficient used in formula (8), with a value range of 0 to 1. Symbol α determines the standard deviation feature of the superpixel, the absolute difference between the number feature of the superpixel and the background model, and the influence proportion in formula (8), which affects the judgment of whether the superpixel contains a moving object.Thr1 is a threshold. After calculating the absolute difference between the pixel points and the cluster centre one by one, these differences need to be compared with the threshold Thr1. If the difference is smaller than the threshold Thr1, it is marked as a background point, and all differences are greater than threshold Thr1 marked as the former scenic spot. The smaller the threshold Thr1, the more sensitive the algorithm, and the easier it is to detect moving objects, but at the same time there will be more noise.Thr2 is a threshold value that calculates the similaritys of the simple histogram by using the Q feature of the number of neighbouring pixel members of the cluster centre. If the degree of similarity s is greater than the threshold Thr2, it is marked as a background; otherwise, it is necessary to further determine the properties of the superpixel. Generally speaking, the larger the threshold Thr2, the better the performance of the algorithm in the experiment, but the computing time will also increase significantly.Thr3 is a threshold, and the weighted similarity measure Dis is calculated by formula (8). If the similarity measure Dis is greater than the threshold Thr3, the superpixel is marked as the foreground; otherwise, it is marked to be updated. Generally speaking, the smaller the threshold Thr3 is, the more sensitive the algorithm is, but if the threshold is too small, it will affect the background update part of the algorithm.K is the number of clustering centres. The larger the K, the more complex the model, and the higher the accuracy, but the algorithm will take more time. Too small K will lead to too many outliers, and it is impossible to construct a scientific background model.Taking F-score as an example, Figure4 shows the experimental results of the F-score value when the parameters α, Thr2, Thr3, and K are set to different values, and other parameters remain unchanged. It can be seen that, in addition to the parameter Thr2, if the values of the parameters α, Thr3, and K change within a certain interval, the F-score value will be higher, and the algorithm performance will be better, and when α, Thr3, and K are set to 0.75, 95, and 30, you can get the best results of the selected parameter experiment. Because the data set is large, it is currently impossible to conduct more detailed experiments, such as considering the simultaneous change of multiple parameters to verify whether there is an impact between the parameters, so, although there may be better parameter choices, the algorithm in this chapter still uses α, Thr3, and k set to 0.75, 95, and 30, respectively, and the other parameters are the same.Figure 4
Experimental results of performance index F-score whenα, Thr1, Thr2, Thr3, and K have different values. (a) F-score value of α and Thr2. (b) F-score value of K and Thr3.
(a)(b)Figure5 shows the F-score value and the running time of the experimental results when the parameter Thr2 is set to different values. The results show that when the Thr2 value gradually increases, the F-score value and running time will gradually increase, but the F-score value will increase. The speed gradually slows down or even falls back briefly, while the running time growth speed gradually increases, and the growth speed increases sharply in the range of 0.8 to 0.9. Therefore, considering the F-score value and running time, the algorithm in this chapter is used in all video sequences. Set the parameters Thr2 to 0.8.Figure 5
The experimental results of performance index F-score and running time when the value of Thr2 is different.


## 4.3. Performance Analysis

FCN has achieved great success in image segmentation. In order to verify whether FCN can be used in the field of moving target detection, experiments and analyses are carried out from the three aspects of the feasibility, reliability, and robustness of the DFCN model, and the DFCN method is compared with mainstream moving target detection algorithms.


### 4.3.1. Algorithm Feasibility Experiment and Analysis

In order to test whether the DFCN model converges, the DFCN model is supervised and trained on the training set for 1000 iterations, and the error of the model on the training set and the test set is recorded. The error curve of the training set and the test set is shown in Figure6.Figure 6
Error curves of the training set, validation set, and test set of the DFCN model.It can be seen from Figure6 that the training set error gradually decreases with the number of iterations and stabilizes after 1000 iterations, and the error is basically zero. The error of the test set decreases with the increase of the number of iterations, but the phenomenon of oscillation appears.This is because the scene where the test set is located is not involved in training at all, and there are many similar pixels between adjacent frames of the scene involved in training. Therefore, for the scenes that have participated in the training, it can be considered that the DFCN model has basically converged, but the model still has a certain error in the scenes that have not participated in the training. In summary, the DFCN method proposed in this paper is feasible.


### 4.3.2. Algorithm Reliability Experiment and Analysis

In order to verify the reliability of the DFCN method proposed in this chapter, the trained network model was tested on a total of 29 scenarios in 7 categories in the data set, and the average value of each evaluation index of the 7 scenarios was recorded with objective evaluation indicators.The average evaluation indicators of the DFCN method in various scenarios on the CDnet dataset are shown in Figure7. It can be seen from Figure 7 that the detection accuracy of the DFCN method in various complex scenarios in the CDnet data set is particularly high, both R and P are higher than 96%, and the comprehensive performance index of the DFCN method in the above complex scenarios is F-score, all are above 96%, and the FPR and FNR are extremely small.Figure 7
The average evaluation index results of the DFCN model in various scenarios. (a) Difference (R) S. (P) and F-score with number of iteration. (b) Difference P-score, FPR, and FNR with number of iterations.
(a)(b)In order to further illustrate the superiority of the DFCN method proposed in this paper over the existing methods, several representative moving target detection algorithms are selected for comparison, including GMM, PBAS, KDE, SACON, SILTP, and SC_SOBS. Figure8 shows the comparison result of the average comprehensive index F-score of different moving target detection algorithms in the 7 types of scenes in the CDnet dataset.Figure 8
F-score evaluation index results of DFCN model and other algorithms in various scenarios.It can be seen from Figure8 that the comprehensive performance index F-score value of the DFCN method in various complex scenarios on CDnet is far superior to other methods. Compared with the latest method, in baseline, bad weather, dynamic background, shadow, camera jitter, and thermal and night videos category scenarios, the average comprehensive performance F-score value has increased by 2.3%, 19.5%, 11.7%, and 6.7%. Therefore, the DFCN method proposed in this paper has been greatly improved.In order to verify whether the input data of different colour spaces affect the detection results of moving targets, the RGB colour space and the HSV colour space are, respectively, selected for experiments, and the average F-score value in various scenarios is tested. The comparison results are shown in Figure9.Figure 9
F-score value comparison of input data using different colour spaces.It can be seen from Figure9 that, in all 7 scene categories in the CDnet dataset, when the input data uses the RGB colour space, the average F-score value of the 4 scene categories is higher than that of the input data using the HSV space. The average F-score value of the three scene categories is lower than the input data using HSV space, and the F-score value is not much different. Therefore, in the DFCN method proposed in this paper, the selection of the colour space of the input data has little effect on the result of moving target detection.


### 4.3.3. Algorithm Robustness Experiment and Analysis

In order to verify the robustness of the DFCN method proposed in this chapter, the trained DFCN model will be tested in other scenarios that are not involved in the training, and evaluation indicators will be calculated. In the experiment, 100% data of 28 videos except the highway video from the 29 videos selected from the CDnet dataset are used for training, and the trained DFCN model is used to test the highway videos that are not involved in the training. The model is in the highway video. The evaluation indicators of the following test results are shown in Table3.Table 3
Evaluation indicators of the DFCN model in the Highway scenario without training.
ScenarioRPSF-scoreFPRFNRP-scoreHighway0.9120.8120.9990.9360.0010.0870.712It can be seen from Table3 that the detection result of the DFCN method in the highway scene without training is not ideal. Among them, the F-score value is 0.936. Moreover, the FNR and P-score in the detection are extremely high, indicating that the DFCN model has a large number of missed detections on untrained highway videos. Experiments have proved that the DFCN method proposed in this paper can get extremely excellent results in the scenes that have participated in the training, but the detection accuracy of the model is not high in the scenes that have not participated in the training. Therefore, the robustness of the DFCN method proposed in this paper still needs to be further improved.


## 4.3.1. Algorithm Feasibility Experiment and Analysis

In order to test whether the DFCN model converges, the DFCN model is supervised and trained on the training set for 1000 iterations, and the error of the model on the training set and the test set is recorded. The error curve of the training set and the test set is shown in Figure6.Figure 6
Error curves of the training set, validation set, and test set of the DFCN model.It can be seen from Figure6 that the training set error gradually decreases with the number of iterations and stabilizes after 1000 iterations, and the error is basically zero. The error of the test set decreases with the increase of the number of iterations, but the phenomenon of oscillation appears.This is because the scene where the test set is located is not involved in training at all, and there are many similar pixels between adjacent frames of the scene involved in training. Therefore, for the scenes that have participated in the training, it can be considered that the DFCN model has basically converged, but the model still has a certain error in the scenes that have not participated in the training. In summary, the DFCN method proposed in this paper is feasible.


## 4.3.2. Algorithm Reliability Experiment and Analysis

In order to verify the reliability of the DFCN method proposed in this chapter, the trained network model was tested on a total of 29 scenarios in 7 categories in the data set, and the average value of each evaluation index of the 7 scenarios was recorded with objective evaluation indicators.The average evaluation indicators of the DFCN method in various scenarios on the CDnet dataset are shown in Figure7. It can be seen from Figure 7 that the detection accuracy of the DFCN method in various complex scenarios in the CDnet data set is particularly high, both R and P are higher than 96%, and the comprehensive performance index of the DFCN method in the above complex scenarios is F-score, all are above 96%, and the FPR and FNR are extremely small.Figure 7
The average evaluation index results of the DFCN model in various scenarios. (a) Difference (R) S. (P) and F-score with number of iteration. (b) Difference P-score, FPR, and FNR with number of iterations.
(a)(b)In order to further illustrate the superiority of the DFCN method proposed in this paper over the existing methods, several representative moving target detection algorithms are selected for comparison, including GMM, PBAS, KDE, SACON, SILTP, and SC_SOBS. Figure8 shows the comparison result of the average comprehensive index F-score of different moving target detection algorithms in the 7 types of scenes in the CDnet dataset.Figure 8
F-score evaluation index results of DFCN model and other algorithms in various scenarios.It can be seen from Figure8 that the comprehensive performance index F-score value of the DFCN method in various complex scenarios on CDnet is far superior to other methods. Compared with the latest method, in baseline, bad weather, dynamic background, shadow, camera jitter, and thermal and night videos category scenarios, the average comprehensive performance F-score value has increased by 2.3%, 19.5%, 11.7%, and 6.7%. Therefore, the DFCN method proposed in this paper has been greatly improved.In order to verify whether the input data of different colour spaces affect the detection results of moving targets, the RGB colour space and the HSV colour space are, respectively, selected for experiments, and the average F-score value in various scenarios is tested. The comparison results are shown in Figure9.Figure 9
F-score value comparison of input data using different colour spaces.It can be seen from Figure9 that, in all 7 scene categories in the CDnet dataset, when the input data uses the RGB colour space, the average F-score value of the 4 scene categories is higher than that of the input data using the HSV space. The average F-score value of the three scene categories is lower than the input data using HSV space, and the F-score value is not much different. Therefore, in the DFCN method proposed in this paper, the selection of the colour space of the input data has little effect on the result of moving target detection.


## 4.3.3. Algorithm Robustness Experiment and Analysis

In order to verify the robustness of the DFCN method proposed in this chapter, the trained DFCN model will be tested in other scenarios that are not involved in the training, and evaluation indicators will be calculated. In the experiment, 100% data of 28 videos except the highway video from the 29 videos selected from the CDnet dataset are used for training, and the trained DFCN model is used to test the highway videos that are not involved in the training. The model is in the highway video. The evaluation indicators of the following test results are shown in Table3.Table 3
Evaluation indicators of the DFCN model in the Highway scenario without training.
ScenarioRPSF-scoreFPRFNRP-scoreHighway0.9120.8120.9990.9360.0010.0870.712It can be seen from Table3 that the detection result of the DFCN method in the highway scene without training is not ideal. Among them, the F-score value is 0.936. Moreover, the FNR and P-score in the detection are extremely high, indicating that the DFCN model has a large number of missed detections on untrained highway videos. Experiments have proved that the DFCN method proposed in this paper can get extremely excellent results in the scenes that have participated in the training, but the detection accuracy of the model is not high in the scenes that have not participated in the training. Therefore, the robustness of the DFCN method proposed in this paper still needs to be further improved.


## 5. Conclusion

This paper proposes a moving target detection method based on a deep full convolutional network of sensor networks. First, a low-power sensor network node design is proposed. The wireless sensor network composed of this node has achieved good performance in the simulation experiment, and low-power consumption can prolong the service life of the battery and reduce the cost. Secondly, the full convolutional network is applied to the field of moving target detection. The background image of the video scene is quickly extracted by the time domain averaging method, and the video sequence and the background image are channel-merged to construct a deep full convolutional network model. Then, the in-depth features of the video scene are learned through the network model to distinguish the difference in detail between the current frame image and the background image, and the pixel-level classification results are output to achieve moving target detection. The deep full convolutional network model proposed in this paper can adapt to complex video scenes of different sizes and achieve pixel-level dense prediction. In the detection process, only one forward calculation is required for each image, and the background extraction method is simple and effective in improving the detection speed.



---

*Source: 3909522-2021-05-20.xml*