# FenceTalk: Exploring False Negatives in Moving Object Detection
## 3 FenceTalk Architecture
These datasets were chosen to encompass a diverse range of lighting conditions and scenarios, enabling us to validate FenceTalk's performance robustness and reliability in real-world outdoor environments. It is noteworthy that due to this high collection frequency, the contents of any two consecutive images exhibited striking similarity. When it came to utilizing these images in training our model, a straightforward approach proved to be counterproductive, as it substantially consumed computational resources without yielding significant benefits to the model's performance. Therefore, we resampled the images at a rate of 5 FPS, which captured one image every 0.2 s. This adjustment allowed us to maintain a sufficiently fast capture rate to effectively track moving objects (people), and train a highly effective model while conserving computational power. Table 1 shows the total number of images utilized in Datasets 1 and 2 after the resampling process. We utilized standard output measures for AI predictions, distinguishing them for the Yolo module, the SSIM module, and FenceTalk (Yolo + SSIM). All images processed by the Yolo module were classified into the following categories: _TP_ (true positives), _TN_ (true negatives), _FN_ (false negatives), and _FP_ (false positives). Therefore, we have  
Figure 2.3  
In the SSIM module, the images in the non-moving object database were classified into _TP\({}^{*}\)_ true positives, _TN\({}^{*}\)_ true negatives, _FN\({}^{*}\)_ false negatives, and _FP\({}^{*}\)_ false positives. Therefore, we have \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset 1** & **No. of Images** & **No. of Images with People** \\ \hline Training & 862,258 & 23,098 \\ Validation & 808,153 & 35,358 \\ Testing & 786,443 & 40,700 \\ \hline
**Dataset 2** & & \\ \hline Training & 496,936 & 22,805 \\ Validation & 479,452 & 20,025 \\ Testing & 402,265 & 19,003 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The images for training, validation, and testing. Figure 8: The locations for data collection. \[\text{SSIM}:\text{ Precision}=\frac{TP^{*}}{TP^{*}+FP^{*}}\text{ andRecall}=\frac{TP^{*}}{TP^{*}+FP^{*}} \tag{7}\]  
and finally, the output measures for \(\text{FenceTalk}\) (Yolo + SSIM) are  
\[\text{FenceTalk}:\text{ Precision}=\frac{TP+TP^{*}}{TP+FP+TP^{*}+FP^{*}} \tag{8}\]  
and  
\[\text{FenceTalk}:\text{Recall}=\frac{TP+TP^{*}}{TP+FN} \tag{9}\]  
Specifically, the total count of all moving objects within a dataset is derived by _TP + FN_. The count of correctly predicted cases by Yolo is represented by _TP_. _TP\({}^{*}\)_ represents the count of accurately predicted cases by SSIM among the FN cases. Consequently, the total count of correctly predicted cases by \(\text{FenceTalk}\) (Yolo + SSIM) is _TP + TP\({}^{*}\)_. Therefore, the recall is calculated as _TP + TP\({}^{*}\)/TP + FN_. The F1 score is expressed as  
\[\text{F1 score}=2\bigg{(}\frac{\text{Precision}\times\text{Recall}}{\text{ Precision}+\text{Recall}}\bigg{)} \tag{10}\]  
In Equation (10), the F1 score takes into account both precision and recall. In \(\text{FenceTalk}\), to find and verify the universality of the optimal SSIM threshold in each dataset, we divided each dataset into three subsets: the training dataset, the validation dataset, and the testing dataset. In the \(\text{FenceTalk}\) experiments, we calculated the precision and recall for both Yolo and SSIM (Equations (6) and (7)). To find the optimal SSIM threshold, we calculated the average grayscale value of each image and used it as an image brightness category. We used an interval of 25 for the SSIM threshold and recorded the _TP_, _TN_, _FN_, and _FP_ counts for different thresholds under various brightness levels. This helped us calculate the precision and recall of SSIM at different thresholds (elaborated in Figures 9 and 10). Since we aimed to collect images that Yolo failed to recognize using SSIM for model training, we selected the threshold with the highest recall value as the optimal SSIM threshold for each brightness level in the experiment.