# Improving CNN Fish Detection and Classification with Tracking
## 6 Conclusions  
In this paper, we showed the possibilities of multiple object tracking [16; 34; 35] to improve fish classification and counting. As the state-of-the-art methods only use the image aspect [25], tracking could be the needed tool to leverage the temporal aspect of the videos. Tracking by detection with a deep learning model proved to be more efficient than any other classic way that involved prediction or statistics [10]. However, a well-trained model can likely miss an object and fail to detect it for a couple of frames if the conditions are changing constantly, such as lighting, turbidity, reflection, etc. For videos recorded underwater, such condition changes are happening frequently, making it difficult to detect the same fish consistently. Adding a tracking module could solve the issue for a handful of frames before the model is able to detect the fish once again. Our goal is to connect previous and future detections of the same fish in case there are periods without detections. This approach enables the continuous tracking of fish under the same ID, as demonstrated by our filtering method. Maintaining the same ID could also enhance classification accuracy by allowing us to correct misclassifications based on the predominant classes assigned to the same detected fish. We anticipated that the model would accurately classify the fish most of the time, with only occasional instances of misclassification occurring within minority classes. However, these instances are expected to be rare compared to the majority classifications. In short, it also gives access to the interaction between frames and thus compensates for a large, acknowledged deficiency, especially in marine data composed of many rare species [36]. Moreover, the coupling of tracking and CNNs with a light architecture make it applicable in real-time applications. Of course, the two-direction tracking (backward and forward tracking) is not applicable for real-time applications, but can be used for long campaigns at sea or for long video dataset analyses. One of the limitations of such a method could occur in highly turbid waters, as with all vision-based methods. Overall, our paper shows that coupling the detection model with a tracking module can improve the detection accuracy of fish in an underwater environment. Figure 7: Example of an early false detection of the fish by the module before it is correctly detected by the model. **Author Contributions:** Conceptualization, B.Z., L.V. and S.V. ; Methodology, B.Z., J.Z., L.V. and S.V. ; Software, B.Z. ; Validation, S.V. ; Investigation, B.Z., J.Z. and S.V. ; Resources, L.V. ; Data curation, B.Z., F.B. and S.V. ; Writing--original draft, B.Z. ; Writing--review & editing, J.Z., F.B., L.V. and S.V. ; Visualization, B.Z. ; Supervision, J.Z., L.V. and S.V. ; Project administration, S.V. ; Funding acquisition, J.Z. and L.V. All authors have read and agreed to the published version of the manuscript. **Funding:** The study was funded by grant ANR \"SEAMOUNTS\" #ANR-18-CE02-0016, the French Oceanographic Fleet, and IRD core funding. The study was supported by the research consortium ANR-AFD \"AIME\". **Institutional Review Board Statement:** Not applicable. **Informed Consent Statement:** Not applicable. **Data Availability Statement:** The raw data supporting the conclusions of this article will be made available by the authors on request, according to the lab politics. **Acknowledgments:** Data were collected under permits 2019-733/GNC, 2020-503/GNC and 2020-1077/GNC delivered by the Government of New-Caledonia, 898-2019/ARR/DENV, 3066-2019/ARR/DENV, 844-2020/ARR/DDDT, and 1955-2020/ARR/DDDT delivered by the Southern Province of New-Caledonia, and 609011/2019/DEPART/JJC, 609011-18/2019/DEPART/JJC, and 609011-39/2020/DEPART/JJC delivered by the Northern Province of New-Caledonia. **Conflicts of Interest:** Author [PERSON] was employed by the company Soproner, Groupe GINGER.