DOI: 10.5061/dryad.f7m0cfz6f
title: Data from: Towards a fully automated underwater census for fish assemblages in the Mediterranean Sea

description: Assessing underwater biodiversity is labour-intensive and costly, but is
 crucial for measuring the extent of the decline in local fish stock. In
 most cases, Underwater Visual Census (UVC) is the preferred method,
 however this can be costly in terms of human effort and is limited by
 meteorological and logistical factors. Advances in technology allows the
 utilisation of more autonomous video recording methods (i.e. Remote
 Operated Vehicles (ROV)) which addresses these limitations. This study
 used a transect-wise UVC coupled with diver operated videos (DOV). For the
 video analysis, a comprehensive fully automated pipeline was developed to
 extract frames from DOV and perform colour correction. This pipeline
 integrates a YOLO-based model to detect 20 Mediterranean fish species and
 validate the presence or absence of each species within individual
 transects. This study was conducted to evaluate the feasibility of using
 video-based methods for UVC with minimal human-input. The result of
 automated video analysis were in agreement with manual video counting,
 validating the autonomous and bias-free procedure for video assessment. In
 conclusion, utilising a minimal-human-input video method liberates the
 data acquisition from limiting factors (i.e. meteorological and
 logistical) and automation of this video analysis significantly reduces
 the labour and time required. For future fieldwork campaigns, the video
 data collection protocol needs to be modified to better resemble
 traditional UVC and enhance this acquisition method.
description: <strong>1. Study area and data
 collection</strong> The training dataset
 (DATA<em><sub>T</sub></em><em> </em>)
 was gathered in eight different locations in the Mediterranean Sea along
 the French Riviera, following the same UVC protocol on each site
 (Harmelin-Vivien <em>et al.</em>, 1985). The depth ranged from
 1-37m and was carried out during the whole year in 2022 (cold and warm
 season) to cover the full range of conditions and possibilities of fish
 occurrences. The experimental dataset
 (DATA<em><sub>E</sub></em>) was recorded in
 October 2023 in and around two protected areas, one no-take zone (Cap
 Roux) and one Natura2000 site (Corniche Varoise), which both have elevated
 biodiversity. The specific coordinates and meta data can be found in the
 supplementary material (Table S1). A total of 64 videos, each
 corresponding to a transect, from 14 sites (8 on seagrass meadows and 6 on
 rocky substrates) were evaluated and compared. Each site consists of 3 to
 6 transects, depending on the availability of video recordings and UVC
 data from the divers. The videos were obtained with
 GoPro HERO 9 cameras, mounted on the clipboards (Fig. 1) used by the divers to note
 the number of fish per species with their respective size category
 (variable number of categories per species). The videos were recorded with
 a framerate of 24 frames per second (FPS) and full high definition
 resolution (1920x1080px). Frames were extracted from these recordings with
 a framerate of 1 FPS for
 DATA<em><sub>T</sub></em><em> </em>and
 5 FPS for DATA<em><sub>E</sub></em>. Fish visible
 for less than 1 second (less than 5 frames) in the videos of
 DATA<em><sub>E</sub></em><em>
 </em>will not be considered in the methodology evaluation as they
 were unlikely to be actual detections. <strong>2.
 Image preprocessing</strong> The frames were
 processed in a next step by a marine biology expert to guarantee correctly
 identified species in the videos. To ensure a good species coverage, 19
 different species and an ’Other’ class were labelled manually in the
 frames resulting in 13,033 images (131 videos in the training set and 47
 independent videos in the test set) in
 DATA<em><sub>T</sub></em><em>
 </em>with a total of 68,573 (train = 40,379, test = 28,194)
 individual fish labels (species breakdown in Table S3) and 8,739 miscellaneous
 labels such as background and diver. The ’Other’ class includes species
 (Table S2) that have
 insufficient occurrences in the test videos (n &lt; 100). Since there
 was a wide range of conditions in the videos, a preprocessing was applied
 to both datasets to enhance each image colour range. For this purpose, a
 pretrained UIEC2-Net model (Y. Wang <em>et al.</em>, 2021) was utilised to enhance
 the images. As the last preprocessing step, images were rescaled to
 960x960px.
description: # Data from: Towards a fully automated underwater census for fish
 assemblages in the Mediterranean Sea
 [https://doi.org/10.5061/dryad.f7m0cfz6f](https://doi.org/10.5061/dryad.f7m0cfz6f) ## Description of the data and file structure ### 1. Study area and data collection The training dataset (DATA_T) was gathered in eight different locations in the Mediterranean Sea along the French Riviera, following the same UVC protocol on each site. The depth ranged from 1-37m and was carried out during the whole year in 2022 (cold and warm season) to cover the full range of conditions and possibilities of fish occurrences. The experimental dataset (DATA_E) was recorded in October 2023 in and around two protected areas, one no-take zone (Cap Roux) and one Natura2000 site (Corniche Varoise), which both have elevated biodiversity. A total of 64 videos, each corresponding to a transect, from 14 sites (8 on seagrass meadows and 6 on rocky substrates) were evaluated and compared. Each site consists of 3 to 6 transects, depending on the availability of video recordings and UVC data from the divers. The videos were obtained with GoPro HERO 9 cameras, mounted on the clipboards used by the divers to note the number of fish per species with their respective size category (variable number of categories per species). The videos were recorded with a framerate of 24 frames per second (FPS) and full high definition resolution (1920x1080px). Frames were extracted from these recordings with a framerate of 1 FPS for DATA_T and 5 FPS for DATA_E. Table 1: Showing the location of where the videos for Data_E were collected as well as site and transect indication. | location | site | transect | videoname | | :--------------- | :--- | :------- | :-------- | | Cap Roux  | 4 | 1 | GH010022 | | Cap Roux  | 4 | 2 | GH010023 | | Cap Roux  | 4 | 3 | GH010024 | | Cap Roux  | 4 | 4 | GH010025 | | Cap Roux  | 6 | 1 | GH010026 | | Cap Roux  | 6 | 2 | GH010027 | | Cap Roux  | 6 | 3 | GH010028 | | Cap Roux  | 6 | 4 | GH010029 | | Cap Roux  | 6 | 4 | GH010030 | | Cap Roux  | 6 | 5 | GH010031 | | Cap Roux  | 6 | 6 | GH010032 | | Cap Roux  | 4 | 1 | GH010396 | | Cap Roux  | 4 | 2 | GH010397 | | Cap Roux  | 4 | 3 | GH010398 | | Cap Roux  | 4 | 1 | GH010399 | | Cap Roux  | 4 | 2 | GH010400 | | Cap Roux  | 4 | 3 | GH010401 | | Cap Roux  | 6 | 1 | GH010403 | | Cap Roux  | 6 | 2 | GH010404 | | Cap Roux  | 6 | 3 | GH010405 | | Cap Roux  | 1 | 4 | GH010417 | | Cap Roux  | 1 | 5 | GH010418 | | Cap Roux  | 1 | 6 | GH010419 | | Corniche varoise | 1 | 1 | GH010446 | | Corniche varoise | 1 | 2 | GH010448 | | Corniche varoise | 1 | 3 | GH010449 | | Corniche varoise | 1 | 1 | GH010451 | | Corniche varoise | 1 | 2 | GH010452 | | Corniche varoise | 1 | 3 | GH010453 | | Cap Roux  | 1 | 1 | GH010736 | | Cap Roux  | 1 | 2 | GH010737 | | Cap Roux  | 1 | 3 | GH010739 | | Cap Roux  | 5 | 1 | GH010740 | | Cap Roux  | 5 | 2 | GH010742 | | Cap Roux  | 5 | 3 | GH010743 | | Corniche varoise | 2 | 4 | GH010757 | | Corniche varoise | 2 | 5 | GH010758 | | Corniche varoise | 2 | 1 | GH011368 | | Corniche varoise | 2 | 2 | GH011369 | | Corniche varoise | 2 | 3 | GH011370 | | Corniche varoise | 1 | 4 | GH011371 | | Corniche varoise | 1 | 5 | GH011372 | | Corniche varoise | 1 | 6 | GH011373 | | Cap Roux  | 2 | 1 | GX010752 | | Cap Roux  | 2 | 2 | GX010753 | | Cap Roux  | 2 | 3 | GX010754 | | Cap Roux  | 2 | 4 | GX010755 | | Cap Roux  | 2 | 5 | GX010756 | | Cap Roux  | 3 | 1 | GX010777 | | Cap Roux  | 3 | 2 | GX010778 | | Cap Roux  | 3 | 3 | GX010779 | | Corniche varoise | 3 | 1 | GX010783 | | Corniche varoise | 3 | 2 | GX010784 | | Corniche varoise | 3 | 3 | GX010785 | | Corniche varoise | 3 | 4 | GX010786 | | Corniche varoise | 3 | 5 | GX010787 | | Corniche varoise | 3 | 6 | GX010788 | | Corniche varoise | 4 | 1 | GX012705 | | Corniche varoise | 4 | 2 | GX012707 | | Corniche varoise | 4 | 3 | GX012709 | | Corniche varoise | 4 | 4 | GX012710 | | Corniche varoise | 4 | 5 | GX012712 | | Corniche varoise | 4 | 6 | GX012714 | ### 2. Image preprocessing The frames were processed in a next step by a marine biology expert to guarantee correctly identified species in the videos. To ensure a good species coverage, 19 different species and an ’Other’ class were labeled manually in the frames resulting in 13,033 images (131 videos in the training set and 47 independent videos in the test set) in DATA_T with a total of 68,573 (train = 40,379, test = 28,194) individual fish labels and 8,739 miscellaneous labels such as background and diver. The ’Other’ class includes 21 species hat have insufficient occurrences in the test videos (n &lt; 100). Since there was a wide range of conditions in the videos, a preprocessing was applied to both datasets to enhance each image colour range. For this purpose, a pretrained UIEC2-Net model was utilised to enhance the images. ### 3. Folder Structure The zip file contains two folders Data_E and Data_T. As described in our publication, the folder Data_T contains images from the training dataset and the folder Data_E contains images from the experimental dataset. Data_T contains two folders – ‘testSet’ and ‘trainSet’ both containing three folders. The test set folder was used in the study to test the DL model and the train folder was used to train the DL model. The folder ‘images’ contains the images. The two labels folder contain labels in the YOLO format (class_id center_x center_y width height). The folder ‘labels_allSpecies’ contains the labels of 40 species labeled with some species being underrepresented. The other folder ‘labels_Reduced’ combines the 21 less important classes into one class called ‘Other’. Data_E contains two folders. The folder ‘images’ contains all the images of the experimental dataset. The folder ‘detections’ contains the DL model predictions from the study with the YOLO format plus the prediction confidence (class_id center_x center_y width height confidence). There are lose files in the root directory. The file called count_classes.py can be used to extract information on the labels of how many of each species are in the datasets. The two files ‘labels_Reduced.txt’ and ‘labels_allSpecies.txt’ give the names of the different classes Data_T was trained on.