## 3 Methods
### Permanent Pixel Extraction Using Spatiotemporal Context Learning
\(P(f(z))\) is the context prior probability that models the appearance of the local context, and \(P(x|f(z))\) represents the relative relationship between \(x\) and its neighbourhood, which is defined as the spatial context model  
\[g_{SC}(x-z)\ =\ P(x|\ f(z)) \tag{4}\]  
Figure 3: Flowchart of permanent pixel extraction procedure. In image #1, \(x*\) and \(z\) are, respectively, the locations of the target pixel and its local context. For context prior probability, when \(x*\) is permanent, if the values of \(x*\) and \(z\), as well as \(I(x*)\) and \(I(z)\), are closer, there is a higher probability that the pixel at location \(z\) will also be permanent. Different from the original solution for the tracking problem, we model the context prior probability \(P(f(z))\) as  
\[P(f(z))=\ e^{-|I(x*)-I(z)|}\omega_{\sigma}(z-x*), \tag{5}\]  
where \(\omega_{\sigma}(z-x*)\) is a spatial weight function. With regard to the attention focus principle, if the local context pixel \(z\) is located closer to the object \(x*\), \(z\) should make a greater contribution to the contextual characteristics of \(x*\) in (5), and a higher weight should be given to it, and vice versa. Given that the weight should decrease smoothly with the increase of the distance to the object, \(\omega_{\sigma}\) is defined as an exponential type as  
\[\omega_{\sigma}(z)=\ ae^{-\frac{|z|^{2}}{2\omega^{2}}}, \tag{6}\]  
where \(a\) is a normalising constant that restricts \(P(f(z))\) to a range from 0 to 1. \(\sigma=0.5\) is the scale parameter. As there are no changes occurring to \(x*\) in image #1, we set its confidence value \(c(x*)=1\). According to the correlation between adjacent pixels, if the context pixel is located closer to the object, it should be more likely to be permanent. Therefore, the confidence function in image #1 can be modelled as  
\[c(x)=P(x)=e^{-|\frac{x-x*}{\alpha}|^{\beta}}, \tag{7}\]  
where \(\alpha\) is a scale parameter and \(\beta\) is a shape parameter. The confidence value changes monotonically with the values of \(\alpha\) and \(\beta\). Therefore, these two parameters can be neither too large nor too small. For instance, if \(\beta\) is too large, the model can easily get over-fitted. While if \(\beta\) is too small, the smoothing may cause some errors. We empirically set \(\alpha=4.5\) and \(\beta=1\) for all the experiments here. Based on (2)-(5), it can be inferred that  
\[ \begin{array}{l}c(x)=P(x)=\sum\limits_{f(z)\in X_{C}}P(x|\ f(z))P(f(z))= \sum\limits_{z\in\Omega(x^{\prime})}g_{SC}(x-z)e^{-|I(x*)-I(z)|}\omega_{\sigma }(z-x*)\\ =g_{SC}(x)\otimes(e^{-|I(x*)-I(x)|}\omega_{\sigma}(x-x*))\end{array} , \tag{8}\]  
where \(\otimes\) denotes the convolution operation. According to (7), (8) can be transformed to the frequency domain as:  
\[F(c(x))=F(e^{-|\frac{x-x*}{\alpha}|^{\beta}})=F(g_{SC}(x))\odot F(e^{-|I(x*)- I(x)|}\omega_{\sigma}(x-x*)), \tag{9}\]  
where \(F\) donates the Fourier transform function. \(\odot\) is the element-wise product. So, for image #1, the spatial context model is  
\[g_{SC}(x)=F^{-1}(\frac{F(e^{-|\frac{x-x*}{\alpha}|^{\beta}})}{F(e^{-|I(x*)-I( x)|}\omega_{\sigma}(x-x*))}) \tag{10}\]  
With the spatial context model gained from image #1, according to (9), the confidence map of image #2 can be calculated by  
\[c^{\prime}(x)=F^{-1}(F(g_{SC}(x))\odot F(P^{\prime}(f^{\prime}(x))))=F^{-1}(F (g_{SC}(x))\odot F(e^{-|I^{\prime}(x*^{\prime})-I^{\prime}(x)|}\omega_{\sigma }(x-x*^{\prime}))), \tag{11}\]  
where \(x*^{\prime}\) is the location of the target pixel in image #2, and \(f^{\prime}(x)\) and \(I^{\prime}(x)\), respectively, represent the context prior probability and image intensity in image #2 [30]. After the permanence confidence map is calculated for image #2, obtained after a flood, we select the pixels with top \(n\%\) confidence values as the final permanent pixels. In this work, we choose \(n=2\). More discussion on how \(n\) influences the result will be given later.