dataesr/scientific-paragraphs-categorization huggingface dataset:
The PS-RBM provides accurate background modeling even in dynamic and noisy environments. Practically, PS-RBM provided similar results than DPGMM  [69] , KDE  [49] , KNN  [234] , and SOBS  [122]  methods on the CDnet 2012 dataset. In a further work, Xu et al. [211]  proposed a Temporally Adaptive RBM (TARBM) background subtraction to take into account the spatial coherence by exploiting possible hidden correlations among pixels while exploiting the temporal coherence too. As a result, the augmented temporally adaptive model can generate more stable background given noisy inputs and adapt quickly to the changes in background while keeping all the advantages of PS-RBM including exact inference and effective learning procedure. TARBM outperforms the standard RBM, and it is robust in presence of dynamic background and illumination changes.
Deep Auto Encoder Networks (DAE) Xu et al. [213]  designed a background generation method based on two auto-encoder neural net-works. First, the approximate background images are computed via an auto-encoder network called Reconstruction Network (RN) from the current video frames. Second, the background model is learned based on these background images with another auto-encoder network called Background Network (BN). In addition, the background model is updated on-line to incorporate more training samples over time. Experimental results on the I2R dataset  [109]  show that DAN outperforms MOG  [180] , Dynamic Group Sparsity (DGS)  [83] , Robust Dictionary Learning (RDL)  [225]  and Online RDL (ORDL)  [121]. In a further work, Xu et al. [212]  improved this method by using an Adaptive Tolerance Measure Thus, DAN-ATM can handle large variations of dynamic background more efficiently than DAN.
Experimental results on the I2R dataset  [109]  confirm this gap of performance. Qu et al. [151]  employed a context-encoder network for a motion-based background generation method by removing the moving foreground objects and learning the feature. After removing the foreground, a context-encoder is also used to predict the missing pixels of the empty region, and to generate a background model of each frame. The architecture is based on the AlexNet architecture that produces a latent feature representation of input image samples with empty regions. The decoder has five up convolutional layers, and uses the feature representation to fill the missing regions of the input samples. The encoder and the decoder are connected through a channel-wise fully connected layer. It allows information to be propagated within activations of each feature map.
The BM-Unet is based on the so called guide features which are used to guide the network to generate the background corresponding to the target frame. Experimental results  [187]  on the SBMnet dataset  12   [100]  demonstrate promising results over neural networks methods (BEWiS  [64] , BE-AAPSA  [155] , and FC-FlowNet  [70] ), and state-of-the-art methods (Photomontage  [1] , LabGen-P  [105] ).
Generative Adversarial Networks (GANs) Generative Adversarial Networks (GAN) have been a breakthrough in machine learning. Introduced in 2014, GAN  [50] [162] provide a powerful framework for using unlabeled data to train machine learning models, rising as one of the most promising paradigms for unsupervised learning. Based on GAN, Sultana et al. [181]   [45]  for classification. Then, the frame selection for inpainting the background is done by summation of pixel values in the forward frame difference technique. If the sum of difference pixels is small, then current frame is selected. Experimental results on the SBM. net dataset  [100]  show that DCP achieved an average gray level error to be 8. 724 which is minimum among all the compared lowrank methods, that are RFSA  [68] , GRASTA  [72] , GOSUS  [210] , SSGoDec  [232] , and DECOLOR  [233].
In a further work, Sultana et al. [182]  extended this method to RGB-D videos by separately training two DCPs: one for RGB videos and one for depth videos. Then, each generated background sample is then subtracted from the given test sample to detect foreground objects either in RGB or in depth. Finally, the final foreground mask is obtained by combining the two foreground masks with a logical AND. Experiments on the SBM-RGBD  13  dataset  [25]  show that ForeGAN-RGBD model outperforms cwisardH+  [65] , RGB-SOBS  [126] , and SRPCA  [90]  with an average F-Measure of 0. 8966.
Background Subtraction Background subtraction consists of comparing the background image with the current image to label pixels as background or foreground pixels. The top algorithms on the large-scale dataset CDnet 2014 are three DNNs based methods (FgSegNet  [113] , BSGAN  [229] , Cascaded CNN  [204] ) for supervised approaches followed by three no-supervised methods that are multi-features/multi-cues approaches (SuB-SENSE  [178] , PAWCS  [179] , IUTIS  [14] ). This task is a classification one, that can be achieved with success by DNN. For this, different methods have been developed in literature and we review them in the following sub-sections. Table  2  shows an overview of these methods. In addition, the list of publications is available at the Background Subtraction Website  14  and is regularly updated.
Convolutional Neural Networks Braham and Van Droogenbroeck  [22]  were the first authors to use Convolutional Neural Networks (CNNs) for background subtraction. This model named ConvNet has a similar structure than LeNet-5  [43]. Thus, the background subtraction model involves four stages: background image extraction via a temporal median in grey scale, specific-scene dataset generation, network training and background subtraction. More precisely, the background model is built for a specific scene. For each frame in a video sequence, image patches that are centered on each pixel are extracted and then they are combined with corresponding patches from the background model. Braham and Van Droogenbroeck  [22]  used a patch size of 27 × 27. After, these combined patches are fed to the network to predict probability of foreground pixels.
With the CDnet2014 dataset (excluding the IOM and PTZ categories), this method with IUTIS and GT achieved an average F-Measure of 0. 7897 and 0. 9046, respectively. Baustita et al. [13]  also used a simple CNN but for the specific task of vehicle detection. For pedestrian detection, Yan et al. [214]  employed the similar scheme with both visible and thermal images. Then, the inputs of the network have a size of 64 × 64 × 8 which includes the visible frame (RGB), thermal frame (IR), visible background (RGB) and thermal background (IR). The outputs of the network have a size of 64 × 64 × 2. Experiments on OCTBVS dataset 15 show that this method outperforms T2-FMOG  [8] , SuBSENSE  [178] , and DECOLOR  [233]. For biodiversity detection in terrestrial and marine environments, Weinstein  [206]  employed the GoogLeNet architecture integrated in a software called DeepMeerkat 16.
9209 in CDnet2014 dataset  [203]. For the Cascaded CNN's implementation 17 available online, Wang et al. [204]  used the Caffe library 18   [98]  and MatConvNet 19. The limitations of Cascaded CNN are as follows: 1) it is more dedicated to ground-truth generation than an automated background/foreground separation method, and 2) it is also computationally expensive. Lim and Keles  [114]  proposed a method called FgSegNet-M 20 based on a triplet CNN and a Transposed Convolutional Neural Network (TCNN) attached at the end of it in an encoder-decoder structure. Practically, the four blocks of the pre-trained VGG-16  [175]  Net is employed at the beginning of the proposed CNNs under a triplet framework as the multiscale feature encoder. Furthermore, a decoder network is integrated at the end of it to map the features to a pixel-level foreground probability map.
Then, a threshold is applied to this map to obtain binary segmentation labels. Practically, Lim and Keles  [114]  generated scene specific models using only a few frames (to 50 up to 200) similar to Wang et al. [204]. Experimental results  [114]  show that TCNN outperforms both ConvNet  [22]  and Cascaded CNN  [204] , and practically outperformed all the reported methods by an overall F-Measure of 0. 9770. In a further work, Lim and Keles  [115]  designed a variant of FgSegNet-M called FgSegNet-S by adding a feature pooling module FPM which operates on top of the final encoder (CNN) layer. In an additional work, Lim et al. [116]  proposed a modified FM with feature fusion. This last version called FgSegNet-V2 21 ranked as number one on the CDnet 2014 dataset. These previous methods usually require a large amount of densely labeled video training data.
To solve this problem, Liao et al. [112]  designed a multi-scale cascaded scene-specific (MCSS) CNNs based background subtraction method with a novel training strategy. The architecture combined the ConvNets  [22]  and the multiscalecascaded architecture  [204]  with a training that takes advantage of the balance of positive and negative training samples. Experimental results show that MCSS outperforms Deep CNN  [4] , TCNN  [226]  and SFEN  [32]  with a score of 0. 904 on the CDnet 2014 dataset by excluding the PTZ category. Liang et al. [111]  developed a multi-scale CNN based background subtraction method by learning a specific CNN model for each video to ensure accuracy, but manage to avoid manual labeling. First, Liang et al. [111]  applied the SubSENSE algorithm to get an initial foreground mask.
Then, an adaptive strategy is applied to select reliable pixels to guide the CNN training because the outputs of SubSENSE cannot be directly used as ground truth due the lack of accuracy of the results. A simple strategy is also proposed to automatically select informative frames for the guided learning. Experiments on the CDnet 2014 dataset show that Guided Multiscale CNN gives a better F-Measure of 0. 7591 than DeepBS  [4]  and SuBSENSE  [178].
Fully CNNs Cinelli  [37]  proposed a similar method than Braham and Droogenbroeck  [22]  by exploring the advantages of Fully Convolutional Neural Networks (FCNNs)  [119]  to diminish the computational requirements. FCNN use convolutional layer to replace the fully connected layer in traditional convolution networks, which can avoid the disadvantages caused by fully connection layer. Practically, Cinelli tested both LeNet5  [43]  and ResNet  [73]  architectures. As the ResNet presents a greater degree of hyperparameter setting (namely the size of the model and even the organization of layers) compare to LeNet5, Cinelli also varied different features of the ResNet architectures to optimize them for background/foreground separation. For this, Cinelli used the networks designed for the ImageNet Large Scale Visual Recognition Challenge (ILSVRC  22  ), which deal with 224 × 224 pixel images, and those for the CIFAR-10 and CIFAR-100 datasets  23  , which have 32 × 32 pixel-images as input.
The FAIR  24 implementation is employed. From this study, the best models on the CDnet 2014 dataset  [203]  are the 32-layer CIFAR-derived dilated network and the pre-trained 34layer ILSVRC-based dilated model adapted by direct substitution. But, Cinelli  [37]  only provided visual results without F-measure. In another work, Yang et al. [216]  also used FCNN but with a structure of shortcut connected block with multiple branches. Each block provides four different branches. Practically, the front of three branches calculate different features by using different atrous convolution, and the last branch is the shortcut connection. For the spatial information, atrous convolution  [75]  is employed instead of common convolution in order to miss considerable details by expanding the receptive fields.
For the activation layers, PReLU Parametric Rectified Linear Unit (PReLU)  [74]  introduced a learned parameter to transform the values less than 0. Yang et al. [216]  also employed a refinement method using Conditional Random Fields (CRF). Experimental results show that this method outperforms traditional background subtraction methods (MOG  [180]  and Codebook  [209] ) as well as recent state-of-art methods (ViBe  [11] , PBAS  [81]  and P2M  [215] ) on the CDnet 2012 dataset  [62]. But, Yang et al. [216]  evaluated their method on a subset of 6 sequences of CDnet 2012  [62]  instead of all the categories of CDnet 2014  [203]  making the comparison more difficult with the other DNN methods. Alikan  [2]  designed a Multi-View receptive field Fully CNN (MV-FCN) based on fully convolutional structure, inception modules  [185] , and residual networking.
9870 while Cascaded CNN  [204]  obtains 0. 8958 whilst MFCN achieves a score of 0. 96 over all the categories. In a further work, Zeng and Zhu  [220]  fused the results produced by different background subtraction algorithms (SuBSENSE  [178] , FTSG  [200] , and CwisarDH+  [65] ) in order to output a more precise result. This method called CNN-SFC outperforms its direct competitor IUTIS  [14]  on the CDnet 2014 dataset. Lin et al. [117]  designed a deep Fully Convolutional Semantic Network (FCSN) for background subtraction. First, FCN is able to learn the global differences between the foreground and the background. Second, SuBSENSE  [178]  algorithm is employed to generate robust background image with better performance, which is concatenated into the input of the network together with the video frame.
Deep CNNs Babaee et al. [4]  proposed a deep CNNs based moving objects detection method which contains the following components: an algorithm for background initialization via an average model in RGB, a CNN model for background subtraction, and a postprocessing module of the networks output using a spatial median filter. First, Babaee et al. [4]  proposed to distinguish the foreground pixels and background pixels with SuBSENSE algorithm  [178] , and then only used the background pixel values to obtain the background average model. In order to have adaptive memory length based on the motion of the camera and objects in the video frames, Babaee et al. [4]  used Flux Tensor with Split Gaussian Models (FTSG  [200] ) algorithm. For the network architecture and training, Babaee et al. [4]  trained the CNNs with background images obtained by the SuBSENSE algorithm  [178].
In addition, batch normalization layers are used before each activation layer to decrease over-fitting and to also provide higher learning rates for training. Finally, a spatial-median filtering is applied in the post-processing step. This method provided foreground mask more precise than ConvNet  [22]  and not very prone to outliers in presence of dynamic backgrounds. Finally, deep CNN based background subtraction outperforms the existing algorithms when the challenge does not lie in the background modeling maintenance. Deep CNN obtained an F-Measure of 0. 7548 in CDnet2014 dataset  [203]. The limitations of Deep CNN are as follows: 1) It can not well handle the camouflage regions within foreground objects, 2) it provided poor performance for PTZ videos, and 3) due to the corruption of the background images, it performs poorly in presence of large changes in the background.
The second network can learn semantic knowledge of the foreground and background. Therefore, it could handle various challenges such as the night light, shadows and camouflaged foreground objects. Experimental results  [226]  show that the TS-CNN outperforms SuBSENSE  [178] , PAWCS  [179] , FTSG  [200]  and Shared-Model  [31]  in the case of night videos, camera jitter, shadows, thermal imagery and bad weather. In CDnet2014 dataset  [203] , TS-CNN and Joint TS-CNN obtained an F-Measure of 0. 7870 and 0. 8124, respectively. In another approach, Li et al. [110]  designed an adaptive deep CNN (ADCNN) to predict object locations in a surveillance scene. First, the generic CNN-based classifier is transfered to the surveillance scene by selecting useful kernels. Secondly, the context information of the surveillance scene is learned in the regression model for accurate location prediction.
Our main contributions. ADCNN achieved very interesting performance on several surveillance datasets for pedestrian detection and vehicle detection but ADCNN focus on object detection and thus not use the principle of background subtraction. Furthermore, Li et al. [110]  provided results with the CUHK square dataset  [199] , the MIT traffic dataset  [202]  and the PETS 2007  25 instead of the CDnet2014 dataset  [203]. In another work, Chen et al. [32]  proposed to detect moving objects via an endto-end deep sequence learning architecture with the pixel-level semantic features. Video sequences are the input into a deep convolutional encoder-decoder network to extract pixel-level semantic features. Practically, Chen et al. [32]  used the VGG-16  [175]  as encoder-decoder network but other architectures, such as GoogLeNet  [185] , ResNet50  [73]  can be also used into this framework.
An attention long short-term memory model named Attention ConvLSTM is used to integrate pixel-wise changes over time. After, a Spatial Transformer Network (STN) model and a Conditional Random Fields (CRF) layer are employed to reduce the sensitivity to camera motion and to smooth the foreground boundaries, respectively. Experimental results  [32]  on the two large-scale dataset CDnet 2014 dataset  [203]  and LASIESTA  [41]  show that the proposed method obtained similar results than Convnet  [22]  with better performance for the category "Night videos", "Camera jitter", "Shadow" and "Turbulence". Attention ConvLSTM obtained an F-Measure of 0. 8292 with VGG-16, 0. 7360 with GoogLeNet and 0. 8772 with ResNet50.
Experimental results  [113]  show that Struct-CNN outperforms SuBSENSE  [178] , PAWCS  [179] , FTSG  [200]  and SharedModel  [31]  in the case of bad weather, camera jitter, low frame rate, intermittent object motion and thermal imagery. Struct-CNN obtained an F-Measure of 0. 8645 on the CDnet 2014 dataset  [203]  excluding the "PTZ" category. Lim et al. [113]  excluded this category arguying that they focused only on static cameras.
3D-CNNs Sakkos et al. [161]  designed an end-to-end 3D-CNN to track temporal changes in video sequences avoiding the use of a background model for the training. 3D-CNN can handle multiple scenes without further fine-tuning on each scene individually. For the architecture, Sakkos et al. [161]  used C3D branch  [190]. Experimental results  [161]  reveal that 3D-CNN provides better performance than ConvNet  [22]  and deep CNN  [4]. Furthermore, experiments on the ESI dataset  [196]  which present extreme and sudden illumination changes, show that 3D-CNN outperforms two designed illumination invariant background subtraction methods that are Universal Multimode Background Subtraction (UMBS)  [160]  and ESI  [196]. 3D-CNN obtained an average F-Measure of 0. 9507 in CDnet 2014 dataset. Yu et al. [218]  employed a spatial-temporal attention-based 3D ConvNets to jointly model the appearance and motion of objects-of-interest in a video for a Relevant Motion Event detection Network (ReMotENet).
For a large number of RPoTP features, the pixel representation is captured even with a small number of ground-truth frames. Experiments on the CDnet 2014 dataset show that DPDL is effective even with only a single ground-truth frame giving similar performance than the MOG model in this case. With 20 GTs, DPDL obtains similar scores than SubSENSE  [179]. Finally, DPDL  26  with 40 GTs gives an average F-Measure of 0. 8106 outperforming DeepBS  [4].
ReLU follows as the activation function in hidden layers. Batch normalization layer and pooling layer are after each ReLU layer. Finally, all feature maps are employed as inputs of a Multilayer Perceptron (MLP) which contains three fully connected layers. Sigmoid is used as activation function and the output only consists of a single unit. Experiments on the SBM-RGBD  27  dataset  [25]  show that BGSNet-D outperforms existing methods that use only depth data, and even reaches the performance of the methods that use RGB-D data.
The convolutions are 3 × 3 spatial filters and their corresponding weights are randomly initialized. Leaky ReLU functions are employed as activation functions. Experimental results on CDnet 2014 datasets shows that BScGAN outperforms ConvNets  [22] , Cascaded CNN  [204] , and Deep CNN  [4]  with an an average F-Measure of 0. 9763 without the category PTZ. Zheng et al. [228]  employed a Bayesian GAN (BGAN) approach. First, a median filter algorithm is used to extract the background and then a network based on Bayesian generative adversarial network is trained to classify each pixel, thereby dealing with the challenges of sudden and slow illumination changes, non-stationary background, and ghost. Practically, deep convolutional neural networks are adopted to construct the generator and the discriminator of Bayesian generative adversarial network.
Stacked Denoising AutoEncoders Zhang et al. [222]  designed a deep learned features based block-wise method with a binary spatio-temporal background model. Based on the Stacked Denoising AutoEncoder (SDAE), the deep learning module learns a deep image representation encoding the intrinsic scene information. This leads to the robustness of feature description. Furthermore, the binary background model captures the spatio-temporal scene distribution information in the Hamming space to perform foreground detection. Experimental results  [222]  on the CDnet 2012 dataset  [62]  demonstrate that SDAE gives better performance than traditional methods (MOG  [180] , KDE  [49] , LBP  [77] ), and recent state-of-art model (PBAS  [81] ). To address robustness against stationary noise, Garcia-Gonzalez  [57]  also used a stacked denoising autoencoders to generate a set of robust features for each patch of the image.
Neural Reponse Mixture Shafiee et al. [166] [167] proposed a Neural Reponse Mixture (NeRM) framework to extract rich deep learned features with which to build a reliable MOG background model. Practically, the first synaptic layer of StochasticNet  [168]  is trained on the Im-ageNet dataset  [45]  as a primitive, low-level, feature representation. Thus, the neural responses of the first synaptic layer at all pixels in the frame is then used as a feature to distinguish motion caused by objects moving in the scene. It is worth noting that the formation of StochasticNets used in the NeRM framework is a one-time and offline procedure which is not implemented on an embedded system. The final formed StochasticNet is transferred to the embedded system. Then, MOG model is employed using the deep learned features.
Motion Feature Network Nguyen et al. [143]  combined a sample-based background model with a feature extractor obtained by training a triplet network. This network is constructed by three identical CNN, each of which is called a Motion Feature Network (MF-Net). Thus, each motion patterns is learned from small image patches and each input images of any size is transformed into feature embeddings for high-level representations. A sample based background model is then used with the color feature and the extracted deep motion features. To classify whether a pixel is background or foreground, Nguyen et al. [143]  employed the l 1 distance. Furthermore, an adaptive feedback scheme is also employed. The training is made with the CDNet 2014 dataset  [203]  and the offline trained network is then used on the fly without re-training on any video sequence before each execution.