# Robust Background Subtraction via the Local Similarity Statistical Descriptor
## 3 Background Modeling
### Background Model Initialization  
Many popular background subtraction methods need a sequence of frames to initialize the background model [12; 30]. However, in some application scenarios, we hope to segment the foreground in few initialization sequences or even from the second frame on. Furthermore, many applications require the algorithm to refresh or re-initialize the background model in the presence of sudden scene changes. Hence, in this paper, we use a single frame (the first frame) to initialize the background model.  
Under the assumption that neighboring pixels share a similar temporal distribution at a given time, the background model of pixel \(p(x,y)\) is initialized by randomly taking the sample features from its neighborhood pixels for \(N\) times as follows:  
\[\mathcal{B}(x,y)=\{F(\bar{x},\bar{y})|(\bar{x},\bar{y})\in\mathcal{N}(x,y)\} \tag{12}\]  
where \(\mathcal{N}(x,y)\) is the neighboring pixel of \(p(x,y)\) and the probability of chosing \((\bar{x},\bar{y})\) follows a 2D Gaussian distribution. In our experiments, a \(7\times 7\) neighborhood region has been demonstrated to be a good choice.  
The number of background samples per pixel was recommended to have a value of \(N=20\) in [20]. In fact, \(N\) is used to balance the precision and sensitively of the model. A larger \(N\) value leads to great precision but lower sensitivity, and vice-versa. Due to the larger representation space inducedby multiple features, we consider increasing the number of background samples and determine the value of \(N\) based on the experiment results performed on the CDnet2012 dataset [28]. As can be seen in Figure 9 (the first subgraph), in some categories, like baseline, camera jitter and shadow, their F-Measure scores tend to be saturated when \(N\) reaches the value of 65. Meanwhile, in some other categories, like thermal and intermittent object motion, with the value of \(N\) increased continuously, the F-Measure score decreases. We can thus find that the value of \(N\) depends on the complexity of the scenarios. In this paper, we set \(N=45\) for all categories. Although the overall F-Measure score tends to reach a maximum when \(N\) reaches the value of 65, larger \(N\) values increase memory and computational complexity, while there is little performance improvement.  
As shown in Figure 6, the background model is initialized as follows. First, the color feature map \(I\) and the LSSD feature map \(LSSD\) are extracted from the first input frame. Then, for each pixel \(p(x,y)\) in the input frame, we randomly select a position \(p(\tilde{x},\tilde{y})\) in its \(7\times 7\) neighborhood region and get the color feature \(I(\tilde{x},\tilde{y})\) from its color feature map \(I\) and the LSSD feature \(LSSD(\tilde{x},\tilde{y})\) from its LSSD feature map \(LSSD\). Finally, the combination of the color feature and the LSSD feature \(\{I(\tilde{x},\tilde{y}),LSSD(\tilde{x},\tilde{y})\}\) becomes a background sample of the pixel \(p(x,y)\). Repeating this process \(N\) times, the background model of \(p(x,y)\) is established. After all the pixels have been traversed, we obtain the final initialized background model. To make it easier to understand, the left bottom of Figure 6 which contains \(N\) color feature maps and \(N\) LSSD feature maps represents the initialized background model. The color feature and LSSD feature from each column represents a background sample of the corresponding pixel.