# Robust Background Subtraction via the Local Similarity Statistical Descriptor
## 3 Background Modeling
### Foreground Detection  
The ViBe algorithm relies on the collection and maintenance of history background samples with a random approach, and determine whether the input pixels fit its background model by counting the number of matches within an intersection threshold. Since our background model integrates multiple features, we proposed a few tweaks to the original algorithm to globally improve our results. Denoting the input frame at time \(t\) as \(\mathbf{I}^{t}\), to classify a pixel \(p^{t}(x,y)\) as foreground or background, we will first calculate the number of matches between the input pixel and its background sample model \(\mathcal{B}(x,y)\). This procedure can be formulated as follows:  
\[M(x,y)=\#\{i|dist(F^{t}(x,y),F_{i}(x,y))<R,i\in[1,N]\} \tag{13}\]  
where \(M(x,y)\) is the number of matches, \(F^{t}(x,y)\) is the input pixel feature, \(F_{i}(x,y)\) is the background sample feature, \(dist(F^{t}(x,y),F_{i}(x,y))\) obtains the distance between the input sample and the given background model sample, and \(R\) is a fixed maximum distance threshold. However, as we know, the input pixel \(p^{t}(x,y)\) contains two features: the color feature and the LSSD feature \(F^{t}(x,y)=\{I^{t}(x,y),LSSD^{t}(x,y)\}\). Hence, we should calculate the distances in two different ways. First, to calculate the similarity between two color features, L1 or L2 distance are the most commonly used metrics due to their simplicity and efficiency [20; 31]. However, based on our experimental results, L2 distance is not only an expensive operation, but is also no better than a simpler L1 distance. Hence, we decided to use the L1 distance to calculate the similarity between color features. If the color distance satisfies:  
\[dist(I^{t}(x,y),I_{i}(x,y))<R_{c} \tag{14}\]  
then a color feature match is found. The color threshold \(R_{c}\) controls the robustness of the background model. A small \(R_{c}\) leads to sensitive foreground detection result, while a larger \(R_{c}\) has better resistance against relevant change, but makes it more difficult to detect foreground objects that are very similar to the background. We also determine the value of \(R_{c}\) based on the experiment results performed on the CDnet2012 dataset [28]. As it can be seen in Figure 9 (the second subgraph), the overall F-Measure score tends to arrive at a maximum when \(R_{c}\) sets the value of 15. Second, to calculate the similarity between two LSSD features, we use a similar strategy with the color features. As we defined in Equation (7), we know that the LSSD feature values distribute in the range of \([0,rc]\). \(rc\) is the number of pixels contained in neighborhood window \(E\), so we define the texture threshold as: \(R_{t}=\delta rc\), where \(\delta\in(0,1)\). If the texture distance satisfies:  
\[dist(LSSD^{t}(x,y),LSSD_{i}(x,y))<R_{t} \tag{15}\]  
then a LSSD feature match is found. According to the experiment results shown in Figure 9 (third and fourth subgraph), we can see that \(\delta=0.2\) and the window size \(r\times c\) set to be \(5\times 5\) can achieve optimal performance. In order to integrate the color feature and LSSD feature into the consideration, in the match-calculating procedure, we first calculate the color similarity. If the color distance is less than the pre-defined threshold \(R_{c}\), then LSSD similarity is calculated. That is to say, to find a match through Equation (13), both the color and LSSD feature must be successfully matched. Figure 7 displays the matching process. After obtaining the number of matches, we take the label of pixel \(p^{t}(x,y)\) as follows:  
\[S^{t}(x,y)= \begin{cases}1,&\text{if }M(x,y)<\#_{min}\\ 0,&\text{otherwise.}\end{cases}  \tag{16}\]  
where 1 means foreground and 0 means background. \(\#_{min}\) is the minimum number of matches required for pixel classification. If the number of matches is larger than or equal to \(\#_{min}\), the current pixel is classified as background, and vice-versa.