## 1 Introduction
[21] applied a simple CNN for vehicle detection. Similarly, the input of the CNN is the observed patch image and the corresponding background patch image. The fully connected layer in the traditional convolutional network is replaced by the convolutional layer, which reduces the amount of computation and the architecture of fully convolutional networks (FCNs) can be trained end-to-end. Encouraged by the recent success of transfer learning and FCNs [22] for semantic segmentation, researchers leveraged FCNs to build background modeling. [PERSON] and [PERSON] [23] proposed a multiscale fully convolutional network architecture which utilized different layer features for background subtraction. [PERSON] [24] varied different features of the ResNet [25] architectures to optimize them for background or foreground separation by exploring the advantages of FCNs. [PERSON] et al. [26] also improved FCNs by using three atrous convolution branches with a different dilate to extract spatial information from different neighborhoods of pixels. [PERSON] et al. presented a convolutional neural network based on a multiscale cascade [27]. In this network, a multiscale CNN model with cascade structure is adopted to model the correlation between adjacent pixels to enhance spatial correlation. The FgSegNet-M20 method [28] is based on a triplet CNN and a transposed convolutional neural network. This method uses the VGG-16 Net to embed an image in multiple scales into the feature space, and, in the decoder part, it uses a transposed convolutional network to learn a mapping from feature space to image space. Subsequent research resulted in the proposed FgSegNet-S and FgSegNet-V2. [PERSON] et al. [29] judged the location of targets in surveillance scenes using an adaptive deep neural network (ADCNN) method. [PERSON] et al. [30] designed a deep learned features based block-wise method with a binary spatiotemporal background model. [PERSON] et al. designed a deep pixel distribution learning model (DPDL) for background subtraction [31]. [PERSON] et al. used an encoder-decoder-structured convolutional neural network for background subtraction [32]. [PERSON] et al. used BGSNet-D to detect moving objects in the scenes where color information was not available [33]. [PERSON] et al. [34] combined background subtraction and CNN for moving objects detection in pumping-unit scene. The aforementioned deep neural network methods have contributed to background subtraction. However, it is evident that they have encountered several limitations. One limitation, for example, is that most of the learning algorithms are supervised methods that are merely used to deal with scene-specific situations. Therefore, for these methods, many historical video data are needed first to train the background model, and then the moving object is detected in the new video. Moreover, the trained model only detects the known targets in the category of the training set, which is an even more stringent limitation of these methods. When the unknown objects in the category appear in the current video scene, it produces incorrect judgments. Another limitation is that many of the algorithms have extremely high computational costs. For example, the deep convolutional neural network is in high demand as a computing resource, and often includes tens of millions of parameters; therefore, the computations required to train this network take a great deal of time and money. Some methods adopt an approach based on patch-wise, a method that segments an image into many patches, and then convolutes each patch separately. However, this approach generates a lot of redundancy. Some other methods adopt a fully convolutional network architecture in semantic segmentation for each pixel. Each pixel is divided into two categories: the foreground and the background. However, the computation load is still very large. Likewise, the storage space required by CNN is also relatively large. In comparison, a traditional background subtraction method (BGS) is computed with a lower computational load.