# FenceTalk: Exploring False Negatives in Moving Object Detection
## 3 FenceTalk Architecture
If the Yolo module detects any objects (Figure 6 (2)), the images containing the moving objects are stored in the moving object database (Figure 6 (3)), and those without the moving object are stored in the non-moving object database (Figure 6 (4)). The process then moves on to the next frame. If the Yolo module does not detect any objects, FenceTalk enters the SSIM module to detect the false negatives images in the non-moving object database. The non-moving object database serves as a buffer due to the differing processing speeds of the Yolo module, which utilizes GPU, and the SSIM module, which operate on CPU. Consequently, the non-moving object database guarantees that all images from the Yolo module can be processed by the SSIM module without any loss of images. FenceTalk calculates the brightness of frame \(f\) as \(l\) and selects the background image BG[\(l\)] with the same or closest brightness to \(f\). The SSIM module then detects missing moving objects by comparing \(f\) and BG[\(l\)] (Figure 6 (5)). If the detection result (Figure 6 (6)) does not contain moving objects, the process continues to the next frame. To reduce false positives caused by changes in background lighting or the addition of new objects to the background, if FenceTalk determines that \(f\) contains moving objects  
Figure 2.2 Operation of the Yolo and the SSIM modules. (Figure 6 (6)), it performs a second check using a secondary background image, \(Bg\). Initially, FenceTalk checks if background image \(Bg\) exists (Figure 6 (7)). If \(Bg\) does not exist (Initially, \(Bg=\) NIL), \(f\) is set as \(Bg\) (Figure 6 (8)) and stored (Figure 6 (9)) for subsequent model training. If \(Bg\) exists, SSIM is again applied to detect moving objects by comparing \(f\) and \(Bg\) (Figure 6 (10)). If the detection result (Figure 6 (11)) does not contain moving objects, the process continues to the next frame. If moving objects are detected, \(Bg\) is replace by \(f\) (Figure 6 (8)), and \(f\) is stored in the missing object database (Figure 6 (9)). The purpose of \(Bg\) is to prevent the repetitive detection and storage of the same moving object across a sequence of consecutive images. The utilization of SSIM(\(f\), \(Bg\)) (Figure 6 (10)) ensures that each moving object is saved in the missing object database only once. After processing all desired image frames, users can retrieve all identified moving objects (Figure 6 (3)) and missing objects (Figure 6 (9)) from the database. These false negative images can be automatically annotated and used to retrain the Yolo model (Figure 6 (12)) to improve its accuracy. The retrained Yolo model can then be used to repeat the recognition process (Figure 6 (1)) for improved recognition accuracy. Figure 7 illustrates the process of automatic background update. The SSIM image (Figure 7 (3)) is generated by comparing the background image (Figure 7 (1)) with the current image frame \(f_{t}\) at time \(t\) (Figure 7 (2)). Noticeable changes in shadows can lead to misjudgment of moving objects by FenceTalk (Figure 7 (4)). Upon detecting moving objects, FenceTalk updates the background image (Figure 7 (5)). The SSIM image (Figure 7 (7)) is generated by comparing the updated background image with the subsequent image frame \(f_{t+1}\) at time \(t+1\) (Figure 7 (6)). The result is a correct judgment that the image frame \(f_{t+1}\) does not contain any moving objects (Figure 7 (8)). City (as shown in Figure 7(b)). The data collection period spanned from 10 December 2021 to 15 December 2022, capturing the full day's camera footage. In Dataset 1, images were collected at a rate of 10 FPS, resulting in a total of 4,832,579 images. Among these, there were 226,518 images containing moving objects (people). Dataset 2 comprised images collected at a rate of 15 FPS, with a total of 6,910,580 images. Within this dataset, 144,761 images were of moving objects. All images had a resolution of 1920 \(\times\) 1080 pixels.