DOI: 10.5061/dryad.1rn8pk11z
title: 3D photogrammetry and deep-learning deliver accurate estimates of epibenthic biomass

description: Accurate biomass estimates are key to understanding a wide variety of
 ecological functions. In marine systems, epibenthic biomass estimates have
 traditionally relied on either destructive/extractive methods which are
 limited to horizontal soft-sediment environments, or simplistic
 geometry-based biomass conversions which are unsuitable for more complex
 morphologies. Consequently, there is a requirement for non-destructive,
 higher-accuracy methods that can be used in an array of environments,
 targeting more morphologically diverse taxa, and at ecological relevant
 scales. We used a combination of 3D photogrammetry,
 convolutional-neural-network (CNN) automated taxonomic identification, and
 taxa-specific biovolume:biomass calibrations to test the viability of
 estimating biomass of three species of morphologically-complex epibenthic
 taxa from in situ stereo 2D source imagery. Our trained CNN
 produced accurate and reliable annotations of our target taxa across a
 wide range of conditions. When incorporated into photogrammetric 3D models
 of underwater surveys, we were able to automatically isolate our three
 target taxa from their environment, producing biovolume measurements that
 had respective mean similarities of 99, 102, and 120% of those obtained
 from human annotators. When combined with taxa-specific biovolume:biomass
 calibrations values, we produced biomass estimates of 88, 125, and 133%
 mean similarity to that of the “true” biomass of the respective taxa. Our
 methodology provides a highly reliable and efficient method for estimating
 the epibenthic biomass of morphologically complex taxa using
 non-destructive 2D imagery. This approach can be applied to a variety of
 environments and photo/video survey approaches (e.g. SCUBA, ROV, AUV) and
 is especially valuable in spatially extensive surveys where manual
 approaches are prohibitively time-consuming.
description: <strong>Biomass regressions</strong>
 Biomass regressions for target taxa were conducted using
 <em>in-situ </em>and <em>ex-situ</em>
 photogrammetry to estimate biovolume and subsequent weighing using dry- or
 wet-weight methods, depending upon the taxa. 
 <strong>Field Biomass Validation</strong>
 Five underwater transects were conducted using photogrammetric
 video surveys. On each transect, three 0.5 quadrats were placed and the
 biovolume of the target taxa was measured from the photogrammetric 3D
 model. All target taxa within each of the quadrats were subsequently
 collected and retained for biomass measurements. Validation of field
 biomass estimates was achieved by comparing the “true” biomass of the
 target taxa (measured from weighing subsampled quadrats) with that
 predicted from our biovolume conversions.  
 Machine-learning 3D annotation validation To
 test the accuracy of the automated model annotation, we compared the
 biovolume from manually annotated meshes with the biovolume from meshes
 created using machine-learning annotated dense clouds. On each transect,
 three separate groups of each of the target taxa were selected to be
 annotated using each method (n = 15 pairs per species). For each of the
 paired models, the biovolume derived from machine learning was normalised
 to that of the manual-annotation biovolume. 
 Machine-learning semantic segmentation A
 training/test/validation dataset was constructed from images taken during
 underwater work and annotated using CVAT annotation software. This was
 used to train DeepLabV3+ and SegFormer frameworks for semantic
 segmentation, using a Python script contained in a Juyter notebook. The
 trained SegFormer framework was used to label images, which were then used
 to label meshes by species. The trained weights for the framework have
 been made available.
description: # 3D photogrammetry and deep-learning deliver accurate estimates of
 epibenthic biomass
 [https://doi.org/10.5061/dryad.1rn8pk11z](https://doi.org/10.5061/dryad.1rn8pk11z) A combination of biomass and biovolume data for NE Atlantic Epibenthic Species and a machine-learning code for automated identification of these species. ## Description of the data and file structure Biomass data is collected in kg or g, dry mass or wet mass. Biovolume is collected using photogrammetric approaches measured in cm^3 or m^3. These data are in Excel format. Data can be used to generate density regressions. # Sharing/Access information This data will also be made available as part of a submission to the NERC British Oceanographic Data Centre. ## Code/Software All training data/code for the machine learning semantic segmentation is contained in a folder called SemanticSegmentation. The training image set is in JPEG format partitioned into train/validate/test sets, and contained in folders with associated names. The Python code is contained in a Jupyter Notebook, 'BiomassMethodsSegmentation.ipynb', that contains annotated steps for training and predicting segmentations. Trained network weights have been made available, named 'swin' and 'deeplabv3+' for two different semantic segmentation frameworks. 'swin' and 'deeplabv3+' files are the saved weights in serialised object format for different trained machine learning segmentation models, that can be opened by the open source PyTorch machine learning library, with example script contained in Jupyter Notebook, 'BiomassMethodsSegmentation.ipynb.