# FenceTalk: Exploring False Negatives in Moving Object Detection
## 3 FenceTalk Architecture
To resolve this issue, we developed the SSIM module. The structural similarity index (SSIM) is a metric used to measure the similarity between two images. It assesses the images based on their brightness, contrast, and structural similarity. It is commonly employed to determine the degree of similarity or distortion between two images. Given a background image \(x\) and a test photo \(y\), both of size \(m\times n\), SSIM is defined as:  
\[\text{SSIM}(x,y)=[\Psi(x,y)]^{\alpha}[c(x,y)]^{\beta}[s(x,y)]^{\gamma} \tag{1}\]  
where  
\[\Psi(x,y)=\frac{2\mu_{x}\mu_{y}+C_{1}}{\mu_{x}^{2}+\mu_{y}^{2}+C_{1}} \tag{2}\]  
\[c(x,y)=\frac{2\sigma_{x}\sigma_{y}+C_{2}}{\sigma_{x}^{2}+\sigma_{y}^{2}+C_{2}} \tag{3}\]  
\[s(x,y)=\frac{\sigma_{xy}+C_{3}}{\sigma_{x}\sigma_{x}+C_{3}} \tag{4}\]  
In \(\Psi(x,y)\), grayscale values of the images are used to compare the similarity in average luminance between the two images. In the function \(c(x,y)\), contrast similarity between the images is assessed by calculating the standard deviation of image pixels. The function \(s(x,y)\) measures the similarity in structural content between the two photographs. In Equation (2), \(\mu_{x}\) and \(\mu_{y}\) represent the average values of the pixel intensities of the two images, while \(\sigma_{x}\) and \(\sigma_{y}\) in Equations (3) and (4) denote the covariance of the two images. Constants \(C_{1}\), \(C_{2}\), and \(C_{3}\) are used to stabilize the function, where \(C_{1}=K_{1}L^{2}\), \(C_{2}=K_{2}L^{2}\), and \(C_{3}=\frac{C_{2}}{2}\). The values of \(K_{1}\) and \(K_{2}\) are set as 0.01 and 0.03, respectively, and \(L\) is the number of possible  
Figure 4: The operation of the Yolo module. intensity levels in the image. For an 8-bit grayscale image, L would be 255. In [20], the values of \(\alpha\), \(\beta\), and \(\gamma\) are set to 1, leading to a simplified SSIM formula of Equation (1):  
\[\text{SSIM}(x,y)=\frac{\left(2\mu_{x}\mu_{y}+C_{1}\right)\left(2\sigma_{xy}+C_{ 2}\right)}{\left(\mu_{x}{}^{2}+\mu_{y}{}^{2}+C_{1}\right)\left(\sigma_{x}{}^{ 2}+\sigma_{y}{}^{2}+C_{2}\right)} \tag{5}\]  
The structural similarity index (SSIM) yields larger values for more similar images and possesses three key properties. The symmetry property is  
\[\text{SSIM}(x,y)=\text{SSIM}(y,x)\]  
The bound property is  
\[\text{SSIM}(x,y)\leq 1\]  
The uniqueness property states that when two images are identical, i.e., \(\mu_{x}=\mu_{y}\) and \(\sigma_{x}=\sigma_{y}\), we have  
Figure 2.0guish background noise from moving objects will be further discussed and experimentally demonstrated in the next section. The flowchart of the Yolo and the SSIM modules is illustrated in Figure 2.1 FenceTalk determines the presence of moving objects by comparing the current frame with a background image. Before processing the image frame detection, FenceTalk selects frames with varying brightness from the video as candidate background images, denoted as BG[\(l\)] (\(0\leq l\leq L=255\)). If the moving objects to be recognized are within the 80 classes of the MS COCO dataset [26], these candidate backgrounds are initially processed by a pre-trained Yolo model trained on the MS COCO dataset to identify images that do not contain the moving object (e.g., people). However, due to limitations in the pre-trained Yolo model's recognition capabilities or when the moving object is not in the MS COCO dataset, users need to examine and remove background images that FenceTalk incorrectly identified as not containing any moving objects. In FenceTalk, moving objects are defined as objects that change their position relative to the background image over time, i.e., they are not fixed in the background. FenceTalk reads a predefined background image BG[\(l\)] at the brightness level \(l\), and from the video, it reads the next frame \(f\) to be processed. Frame \(f\) first undergoes the Yolo module for object recognition (Figure 6 (1)), where the Yolo model can be a pre-trained model on the MS COCO dataset or a user-trained model.