## 4 Evaluation  
In this section, we compared RoiSeg with the traditional foreground segmentation algorithms. Because the CNN foreground segmentation algorithms are not suitable for these scenarios due to their strict real-time requirements. [PERSON] tested and compared 29 background subtraction algorithms and recommended five of the best, namely DP-WrenGABGS, MixtureOfGaussianV1 BGS, MultiLayerBGS, PixelBasedAdaptiveSegmenter and LBAAdaptiveSOM [38]. In this paper, we used these five algorithms to compare with our proposed method and the hardware of our experiment is a Lenovo desktop with Intel(R) Core(TM) i5-4590 CPU @ 3.3 GHz, 8 GB RAM, Win 10 64 bit system. Because the foreground detected by frame difference has an aperture, we manually filled some aperture in the foreground in order to evaluate our algorithm using \(P\), \(R\) and \(F\). As the size of the self-captured sequences \"My_video1\" and \"My_video2\" were 1280 \(\times\) 720 and that of the sequences provided by the BMC database was 640 \(\times\) 480, the FPS we give was the average for all test sequences. In Section 3.2, for the video sequences \"112\", \"122\", \"212\", \"222\", \"312\", \"322\", \"412\", and \"422\", we found that after filtering with the preset threshold, all foregrounds were recognized. Thus, we did not need a classifier to distinguish foreground and background. The results of running these five algorithms and the proposed RoiSeg on these eight sequences are shown in Table 4 and Figure 13. The average \(P\), \(R\) and \(F\) of the five algorithms and proposed RoiSeg were computed, as shown in Table 4. We could see that the proposed RoiSeg had the best \(R\) and the best \(F\) on some sequences. For video sequences \"112\" and \"122\", which were without noise, and \"212\" and \"222\", which had salt and pepper noise, the proposed noise filter method did not function best; however, the \(R\) and \(F\) still reached over 0.9. For video sequences \"312\" and \"322\", which having moving cast shadows, and \"412\" and \"422\", which were foggy, the proposed noise filter method functions best. A non-optimal \(P\) indicated that our method produced more false positives than other algorithms. The reason for this was that the foreground detected by frame difference has ghosts, and the increased false positives were mainly located at the boundaries of moving objects, which was not harmful for visual observation. From Table 4, we found the proposed noise filter method had the highest FPS. This is due to the effectiveness of the proposed ROI-based noise filter, which mainly focuses on the ROI instead of the full frame. For the video sequences \"My_video1\", \"My_video2\", \"512\", and \"522\", with wind and camera jitter, the noise could not be removed by filtering. Thus, we needed a classifier to distinguish foreground and background. The results of applying these five algorithms  
Figure 12: Experimental results from the Naive Bayes with the balanced and updated training sets. Top to bottom: results for “My_video1”, “My_video2”, “512”, “522”. and the proposed RoiSeg algorithm are shown in Table 4 and Figure 14. From Figure 14, we could observe that the proposed RoiSeg algorithm produced better visual results than the five algorithms. Furthermore, Table 4 shows that the RoiSeg algorithm had the best \(F\), which proved that our method had the best overall performance. After the proposed clustering and online classification work in Sections 3.3 and 3.4, the FPS of our method decreases. However, Table 4 shows that the FPS of our method still outperformed those of the other approaches. We also evaluated the performance of RoiSeg on different datasets with the metrics of the average pixel error rate (APFPER) and the joint intersection overlap (IoU) [39]. APFPER measured the number of misclassified pixels and IoU was to calculate the combined intersection of the estimated and true split plots for evaluating the split performance.