# Improving CNN Fish Detection and Classification with Tracking
## 1 Introduction  
At a time when anthropogenic activities and global changes are exerting increasing pressure on marine ecosystems [1] the enhancement of underwater wildlife management and conservation has become more critical than ever [2]. The regular and consistent monitoring of these ecosystems is essential for detecting changes over time, understanding the ecosystems' health and functionality [3] and informing effective conservation strategies [4]. Traditional methods of underwater monitoring often prove to be labor-intensive [5], expensive [6], and sometimes lack the desired accuracy or resolution. Therefore, it is necessary to design innovative tools to monitor marine biodiversity frequently and on a large scale. Recent advances in underwater camera technology have facilitated the collection of an enormous amount of marine imagery [7]. However, given that the manual processing of such vast data is impractical due to constraints of time and resources, algorithms based on Machine Learning (ML) and deep learning (DL) have been developed to automate the data processing [8]. These algorithms can identify and classify underwater species [9; 10], map habitats [11], and track movements [12] and behaviors [13], thereby creating extensive databases of information that can be analyzed to identify patterns and trends. Despite their promise, these algorithms currently face limitations, such as the need for large training datasets [14], challenges with low light or turbidity, difficulties in identifying rare or camouflaged species, and the need for ongoing validation and refinement. Furthermore, Convolutional neural networks (CNNs) are designed to analyze each image separately while data often comprises video formats with similar information between frames. As such, algorithms are not leveraging the temporal aspect of the video. One way to enhanceautomatic algorithms for counting and identifying fish could be to exploit this information by linking the detection models, like CNNs, with a tracking algorithm [15]. When dealing with tracking algorithms for multiple objects [16], the process entails monitoring objects within a video. This is achieved by analyzing each frame separately, conducting object detection on each frame, and subsequently attempting to establish correspondences between the objects detected in the current frame and those identified in the preceding frame. This matching process is facilitated by employing a specific measurement technique designed to enhance the accuracy of the object matching. If fish tracking in video has been a topic for a decade [17; 18; 19], the coupling of deep learning algorithms and tracking algorithms is a recent and current topic. CMFTNet [20] employs a deformable convolutional network backbone architecture combined with a counterpoised loss function to effectively detect and track individual fish in aquaculture videos. Its performance is evaluated using metrics such as MOTA and IDF1, although these do not include ecologically relevant metrics. [PERSON] et al. [21] combined YOLOv5 with an adaptation of SiamRPN++, originally designed for single target tracking. The paper proposed a use case of this method to analyze the behavior of the porphyry seabream in aquaculture videos. FishMOT [22] proposed to couple a YOLOv7 detection algorithm with IoU evaluation serving as the tracking module. The algorithm is applied to Zebrafish seen from above, using the Trex 2D scenery [23] and the idtracker.ai [24] datasets. To our knowledge, this is the first paper proposing a coupling between a deep architecture and a tracking algorithm applied in unconstrained baited remote underwater videos. This process heavily depends on the accuracy of the detector. However, fish detection models are not usually able to be consistent and fail to detect that same object for a couple of frames [25], leading to extra ID assignments and thus a low tracking quality and poor counting quality. In our paper, we propose an end-to-end method consisting of a CNN detector, a tracker, and post-processing.