Marine cabled video-observatories allow the non-destructive sampling of species at frequencies and durations that have never been attained before. Nevertheless, the lack of appropriate methods to automatically process video imagery limits this technology for the purposes of ecosystem monitoring. Automation is a prerequisite to deal with the huge quantities of video footage captured by cameras, which can then transform these devices into true autonomous sensors. In this study, we have developed a novel methodology that is based on genetic programming for content-based image analysis. Our aim was to capture the temporal dynamics of fish abundance. We processed more than 20,000 images that were acquired in a challenging real-world coastal scenario at the OBSEA-EMSO testing-site. The images were collected at 30-min. frequency, continuously for two years, over day and night. The highly variable environmental conditions allowed us to test the effectiveness of our approach under changing light radiation, water turbidity, background confusion, and bio-fouling growth on the camera housing. The automated recognition results were highly correlated with the manual counts and they were highly reliable when used to track fish variations at different hourly, daily, and monthly time scales. In addition, our methodology could be easily transferred to other cabled video-observatories. Recent technological progress has rapidly advanced the exploration of the world's oceans, opening up new possibilities to address questions related to the variety, distinctiveness and complexity of marine life. Nevertheless, many of the existing technologies still have to be fully transferred to marine sciences. This is particularly needed to develop innovative systems for biological monitoring, to implement them, and to evaluate their performance 1 . Computer vision and machine learning methodologies are expected to offer new tools for use in the marine sciences, and they will have a wide range of applications in ecology and ecosystem management, such as stock assessment and species conservation 2 . As human impacts and global climate change accelerates, one of the most urgent tasks for the coming decades is to develop technologies to continuously track and accurately predict biological responses, which will provide solid guidelines for their management and conservation 3 . The traditional concerns of marine research and applications have been in quantifying the abundance of species through space and time, and also in understanding patterns of fish behaviour 4 , such as the fisheries management 5 and aquaculture 6 . Changes in fish communities, especially regarding commercial species, are also considered under relevant international management policy actions, such as in the case of the EC Marine Strategy Framework Directive (2008/56/EC). In this management effort, cameras are increasingly being considered as one of the most promising approaches for biodiversity monitoring 7,8 . Video-information coupled to concomitant environmental monitoring is particularly needed because some species can rapidly shift their distribution according to environmental drivers 9,10 with complex alterations of the associated ecosystem services 11, 12 . With respect to the traditional sampling approaches, video-based fish counts are gaining popularity as an effective, non-invasive sampling method in the marine environment 13 . This technology has been proposed as a valuable and cost-effective complement to expensive in situ monitoring programs that are operated through vessels (e.g., trawling and ROV surveys) 14 . Finally, increasing efforts are being made to implement the use of underwater video cameras. Published: xx xx xxxx OPEN www.nature.com/scientificreports/ 2 Scientific REPORTS | (2018) 8:13748 | Despite their high deployment and maintenance costs 15 , installing cameras coupled with other biogeochemical and physical sensors allows cabled observatories to provide powerful devices for quantifying biotic components at time frequencies that span from seconds to hours, months, and even years, producing a huge amount of data that urgently needs appropriate methodologies for an effective automated processing 16 . The resulting dataset can be used to establish solid cause-effect relationships between biotic responses (e.g., macro-and megafaunal community changes) and environmental perturbations of either natural or anthropogenic nature. This continuous and coupled observation of biological and environmental parameters represents the core of an ongoing "technological transition", which will have significant implications for future monitoring strategies 17 . Computer vision and pattern recognition are key elements of this technological progress and they offer new possibilities to use marine cabled video-platforms. Over the last two decades, a number of methodologies have been proposed for fish species recognition 18 . However, the great variability arising from either divergent species morphologies or from fluctuating conditions in which the videos are captured is still a major challenge for automated processing 4 . Indeed, the methods that have been developed so far have the main shortcoming of having been tested under controlled conditions, such as constrained environments (e.g., fish farms or laboratories) or stable-optimal meteorological conditions (i.e., good water transparency and no or very low turbidity) 19 or in relation to the classification of single species 20,21 . Many automated recognition and classification approaches have been experimented and validated on the Fish-4K-knowledge (F4K) repository 22 , which only provides underwater images acquired during the daylight (i.e., from the sunrise to the sunset) in oligotrophic and transparent coral reef waters. These automated approaches span a wide range of topics, from statistics 23 to convolutional neural networks 13, 24, 25 and unsupervised machine learning 26 . These methods are used to recognise, classify, and count fish specimens. Other approaches have been validated through the ROV images that are acquired in the deep-sea light homogeneous environment 27 or in aquarium trials 28 . In addition, ad-hoc aquaculture devices have been employed to force the fishes to swim frontally to the video cameras 6 . Methods that are robust enough to handle all of the possible varying conditions of the natural environment are still unavailable 2,4,29 but are highly requested to transform the cameras on underwater cabled observatories into quantitative sensors for ecological monitoring 30 . In this study, we have developed a novel video-automated procedure to effectively track and estimate fish count variations, without discriminating among different species, in a challenging real-world scenarios. A general supervised machine learning framework for image content-based fish recognition was conceived and evaluated under different light conditions, variable water turbidity, and changing bio-fouling coverage on the camera housing. We used a K-fold Cross-Validation framework 31 to select the most relevant image-features 32 and to produce an automated image classifier with high generalization performance 33 . Tests were performed on more than 20,000 images that were acquired at the OBSEA EMSO testing-site 34 during the years 2012 and 2013, at 30 min frequency, continuously over day and night. The fish counts and the related time series were validated by comparing automated versus manually generated data.
Focusing only on the fish detection task, REF reported a method of automatic fish counting in real-world videos, which is referred to as the OBSEA method hereafter.
Tracking Fish Abundance by Underwater Image Recognition
Scientific Reports
Scientific Reports
Medicine Computer Science