# Towards an open digital catalogue of sound signatures
from the underwater soundscape

Timothée Maison, Camille Blayo, Yaëlle Pihan, Myriam Lajaunie, Laura

Ceyrac, Hélène Pihan-Le Bars

► To cite this version:

Timothée Maison, Camille Blayo, Yaëlle Pihan, Myriam Lajaunie, Laura Ceyrac, et al. Towards an open digital catalogue of sound signatures from the underwater soundscape. Inter-Noise 2024: the 53rd International Congress and Exposition on Noise Control Engineering, Aug 2024, Nantes, France. hal-04897167

HAL Id: hal-04897167

https://hal.science/hal-04897167v1

Submitted on 28 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Towards an open digital catalogue of sound signatures from the underwater soundscape

Timothée Maison<fnref n="1" />, Camille Blayo<fnref n="2" />, Yaëlle Pihan<fnref n="3" />, Myriam Lajaunie<fnref n="4" />, Laura Ceyrac<fnref n="5" />
Service Hydrographique et Océanographique de la Marine
13 rue du Châtellier, 29200 Brest, France

Hélène Pihan-Le Bars<fnref n="6" />
Service Hydrographique et Océanographique de la Marine
13 rue du Châtellier, 29200 Brest, France
Institut de Recherche de l'École Navale
CC 600, 29240 Brest Cedex 9, France

## ABSTRACT

Passive acoustic monitoring plays a crucial role in the study the underwater soundscape. Identifying the sound signatures of specific marine species and anthropogenic contributions is a key step in assessing the impact of human activities on the acoustic environment. The project COSS (Catalogue of Sound Signatures) aims to build an open library of sound signatures from the coastal seascape of European waters.

In this framework, a data management workflow is developed within FAIR principles from the storage of the metadata, ensuring integrity and traceability of the recorded datasets, to the analysis and the annotation method, providing interoperability classification of sound signatures. An interactive map provides an overview of the database. Sound events are analyzed using a home-made annotation software. Labeling is based on a tree-structure ontology with respect to the biological species, human activities and geophony. This labels standardization makes it possible to extract information at several levels of precision.

In particular, this library will serve as a learning and testing base for open source AI algorithms for the automatic classification of signals of anthropogenic origin. A long-term objective is to provide a means of determining the acoustic impact of human activities on the marine environment.

## 1. INTRODUCTION

Recordings of underwater soundscapes provide crucial information about the health of the environment and the presence of species. Soundscapes can be classified in their globality to

<footnote n="1">¹timothee.maison@ens-lyon.org</footnote>

<footnote n="2">²camille.blayo@shom.fr</footnote>

<footnote n="3">³yaelle.pihan@shom.fr</footnote>

<footnote n="4">⁴myriam.lajaunie@shom.fr</footnote>

<footnote n="5">⁵laura.ceyrac@shom.fr</footnote>

<footnote n="6">⁶helene.pihan-le.bars@shom.fr</footnote>
facilitate their identification [1]. The sounds contributing to a soundscape are usually categorized into three components: the geophony (abiotic activities), the biophony (biotic activities) and the anthropophony (human activities) [2]. Associating each sound signature to a specific sound source, as a unique acoustic patterns regarding the spectral and temporal characteristics, is essential for classifying the complex sound mixture in soundscape recordings [3]. A dataset of biological species categorized by their known sonifery has recently been proposed [4], but a focus on anthropophony is still needed to study the impact of human activities [2]. Particularly, shallow seas are highly dynamic areas, with intensive human activities [5]. Most recordings and studies have been conducted on shallow ecosystems (depth less than 50 m) due to accessibility constraints [6]. In the framework of the implementation of the European Union Marine Strategy Framework Directive 2008/56/EC, descriptor 11 ("introduction of energy, including underwater noise, is at levels that do not adversely affect the marine environment"), the project COSS (Catalogue of Sound Signatures) aims to build an open-access library of sound signatures from the underwater soundscape in shallow European seas. Sound reference signals completed by information about their provenance and detailing their different parameters open up possibilities for the valuation of underwater soundscape data: ranging from potential automated environmental monitoring to offshore safety applications. The data catalogue, following the guideline of FAIR principles (Findable, Accessible, Interoperable and Reusable) [7], will be integrated in the structure of the European Marine Observation and Data Network (EMODnet). They will also served to train open-source artificial intelligence algorithm in order to achieve automated signal classification. Such a tool will finally provide a means to determine the acoustic impact of human activities and enable the identification of measures to manage these activities in a way that reduces their potential harm to marine life.

This article presents the preliminary and ongoing work at Shom on setting up the library, which does not constitute the final european project outcome and overview. A first section describes the storage architecture selected for data and metadata, and discusses the choices made with regards to the existing datasets and standards. A second section presents data pre-processing and pre-analysis tools used to detect the best data to prioritize in the manual annotation work, that is the scope of the third section.

## 2. STORAGE OF DATA AND METADATA

Collection of audio recordings is often performed during dedicated campaigns lasting several days, weeks or even months. As technology advances and equipment costs fall, data volumes continue to grow. It results in the production of terabytes of audio data which need to be properly stored and tracked.

To fulfill the FAIR principles, data are stored in simple storage service (S3), ensuring high scalability, durability, high availability and ease to manage properties [8]. To optimize storage efficiency, all files are converted to FLAC (Free Lossless Audio Codec) format<fnref n="7" />. FLAC is an universal audio format which conserves the original audio flux and compressed the file without reducing the quality of audio files. This conversion saves up to 60% of the initial volume, increasing storage capacity.

Efficient identification of audio file is crucial when dealing with large amount of data. The metadata associated with the audio file are essential for future identification, tracking, analysis or sharing. A metadata convention has been proposed for passive acoustic recordings which describes a list of necessary metadata, along with their definition [9]. Alternatively, an open-source, portable scientific workbench, called Tethys, has been set up and can be used by individual entities [10]. Tethys provides a solid structure of metadata standard, as it was built to handle a large variety of recording systems and types of survey. Metadata are stored in an XML

<footnote n="7">⁷https://xiph.org/flac/index.html</footnote>
database [11] and can be displayed on a world map based on their GPS coordinates.

The choices outlined in this document are based on these proposed standards for metadata definitions. However, with a view to data sharing, the number of metadata elements has been reduced to focus on the identification and analysis process. Also, the XML structure is less optimized for access speed considering the large volume of data generated and becomes less popular among users. We have preferred to link each audio file to its metadata in a PostgreSQL relational and open source database, used for monitoring complex system and optimized for quick data lookups and updates [12].

A homemade web interface has been implemented to navigate into datasets. This application is developed in Python whose libraries operate well with S3 storage. It converts audio files to FLAC format and pre-analyzes them (see Sec. 3) when they are uploaded into the S3. Additionally, the user is provided with a form with key metadata to fill, such as location, period of recording, and measurement system configuration (non-exhaustively). Some technical metadata, i.e. concerning the audio signal itself (duration, bit depth, sampling frequency), are also extracted from the file header for pre-analysis and quality control. Also the FLAC format allows to save embedded metadata to the audio file to ensure a long-lasting identification, by means of Python library Mutagen<fnref n="8" />. The complete data management is presented in Fig. 1, depicting the different layers of accessibility from raw audio file to its associated metadata, and the ability to easily retrieve the targeted information for analysis.

<figure><img src="image_3.png" /><figcaption>Figure 1: Data management from uploading the audio file to the S3, to final export as CSV of a catalog of metadata.</figcaption></figure>

Along with the web interface, an interactive map shows the location of the datasets, together with their key metadata, such as recording duration and number of acquisition system used (see Fig. 2). This map can be enhanced with open source contextual data (e.g. from EMODnet) to give a first overview of the recording context, i.e. about possible sound events that could be found in the considered recordings.

<footnote n="8">⁸https://mutagen.readthedocs.io/en/latest/</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 2: Screenshot of the interactive map showing location of recorder deployment (red dots), zoomed to the french west coast. Key metadata are highlighted and can be further displayed by clicking on each deployment.</figcaption></figure>

### 3. PRE-PROCESSING AND DATA CONTENT ANALYSIS

Since analysis and labeling is a tedious and time-consuming process that requires expertise and training, it is useful to gather information on audio content from automated algorithms. A first approach is to estimate the quality of the audio content by detecting saturation and computing the Signal-to-Noise Ratio (SNR). Saturation is typically encountered when the sound pressure level has exceeded the maximum input of the recording system, leading to clipping of the audio signal. The SNR is computed from the hydrophone noise floor reference, giving an insight of the quality recording and the audio content. These indicators ensures recording coherence and enable to put aside meaningless audio file, saving time to annotators.

Secondly, pre-analysis of audio content and detectors of sound signatures can guide annotation choices. Knowledge of ambient noise levels over different frequency bands and the application of noise separation enable us to compute sets of parameters that provide information on the relevance of the data, such as the presence probability of signals of interest (relative to the ambient noise) in a time-frequency representation (the percentage of bins in the spectrogram that contain a signal). An example of statistical analysis is illustrated in Fig. 3.

<figure><img src="image_5.png" /><figcaption>Figure 3: Example of statistical detector: (left) power spectral density of a recording of ship signature, airgun and sediment sounder sonar; (center) enhanced power spectral density, i.e. without ambient noise; (right) signal presence probability.</figcaption></figure>

Inspired by specific detectors of sound signatures [13, 14], we are also developing detectors for:
- anthroponegenic sources: boats (diesel engine signature and modulation of broadband noise by the propeller), short transient signatures of the air-gun type

- biological sources: short transients (sperm whale and dolphin echolocation), long transients (dolphin and whale tonal emissions)

This is a first step before the implementation of methods based on machine learning. These expert and statistical detectors should also constitute benchmarks for their evaluation.

## 4. MANUAL ANNOTATION METHOD

Labeling sound signatures is a key step, both for conducting studies on passive acoustic monitoring and for training artificial intelligence algorithms (in supervised and semi-supervised cases). Automated detection and classification methods are fast and objective (with controlled parameters) but are often designed to deal with a specific signal, and subject to variable results. On the other hand, manual processing is more accurate but time-consuming, and subjective because there is no standardized annotation protocol [15]. Currently, no fully automated process can replace an expert annotator considered as a reference, even though manual annotation is impacted by several factors such as the SNR, the annotator profile or the type of acoustic events [16].

In the framework of the data management (see Sec. 2) and to enable manual annotation, a lightweight web application directly linked to the database has been developed. Similarly to most annotation software, sound signatures are analyzed on a spectrogram, as the most used and time-frequency representation, with the help of automated detection and pre-processed information, if applicable (see Sec. 3). The annotation framework has been designed to leave the annotator free to set several parameters for the spectrogram computation and display because the sound signatures to be annotated can cover very different time and frequency ranges: from low-frequency passing ships during dozens of minutes to impulsive broadband echolocation clicks. Sound signatures are detected and defined by a time-frequency box (start time, end time, minimum frequency, maximum frequency) manually drawn on the spectrogram, which is an easy way to segment the audio content on an image [17].

Each sound signature is identified with a source label which describes the source of the sound. Most passive acoustic monitoring studies target specific biological species and use labels designed for the study. Building up a sound library involves confronting and annotating a wide variety of sources, with the need for a meaningful ontology with interoperable labels. Therefore, regarding the biophony, we based our ontology on key parts of the phylogenetic tree of life. It enables to classify according to order, micro-order and family of animals rather than directly to species, as identifying a species can sometimes be very complex. The anthropophony ontology is also designed on a tree-structure, populated by the activity producing the sound (e.g. Marine Renewable Energy production), the sound source (e.g. Windmill), the two being interdependent [5], or even by the characteristics of the sound source (e.g. Foundation type of the windmill). Two branches of the ontology illustrating the tree structure of biophony and anthropophony are shown in Fig. 4.

Such a classification is scalable (branches can be added if necessary) and allows a source to be labeled at several levels of the ontology depending on the precision of the identification and the expertise of the annotator. From the analysis point of view, it is also possible to extract and use labeling information at several levels in the tree structure, e.g. if studying Mysticetes in general or Blue Whale specifically. Moreover, when possible, we add a type label which describes the nature of the signal of the sound signature, e.g. stationary, transient or impulsive. This type label enables to differentiate sound signatures coming from the same source but not the same signal characteristic, e.g. frequency modulated whistles and impulsive echolocation clicks of Bottlenose Dolphins that share a same sound label but have different type labels. In the same way, it can
<figure><img src="image_6.png" /><figcaption>Figure 4: Examples of two branches of the ontology illustrating the tree structure of biophony, based on the phylogenetic tree of life, and anthropophony, based on human activities.</figcaption></figure>

be a supplementary information to design and evaluate AI classifiers. Annotation tables, with time-frequency boxes and labels, are saved in the PostgreSQL database, along with the metadata associated to the audio file (see Sec. 2).

## 5. CONCLUSION

In this article, we introduced the COSS (Catalogue of Sound Signatures) project, which aims to build an open library of sound signatures from shallow European seas, with a focus on anthropophony. We describe work in progress on the contribution and developments made at Shom in relation to the project, which are not the final outcome of the project. Data management choices have been made in line with FAIR (Findable, Accessible, Interoperable and Reusable) principles: simple storage service (S3), conversion of audio to FLAC format to increase storage capacity, metadata saved in a PostgreSQL database. Metadata, inspired from previous works, have been optimized in a data-sharing point of view, focusing on identification and analysis. Saturation detection and Signal-to-Noise Ratio are used to control audio quality. Statistical analysis and specific detectors are developed to guide and save time on the manual annotation work, which remains the reference when performed by experts. The labeling of sound sources is based on an evolutionary tree structure, that enables annotation at several levels of precision, using the phylogenetic tree for biophony and human activities for anthropophony. The type of sound is also labeled to make the difference between the signal characteristics of several emissions from a same source. Such labeling is essential to train future AI algorithms that could manage the quantity of data continuously growing, with efficient detection and classification. The sound signature library could be used to compare and analyze data from other research studies, as well as providing a tool for assessing the impact of anthropogenic activities on underwater soundscapes.

## ACKNOWLEDGEMENTS

This research was funded, in whole or in part, by the European Climate, Infrastructure and Environment Executive Agency (CINEA), under the contract CINEA/CD(2022)5010/PP/SI2.899121. We also acknowledge the French Ministry of Ecological Transition (MTECT) for supporting and funding this work under the convention concerning the mission of scientific and technical support for the implementation of the Marine Strategy Framework Directive (MSFD).

## REFERENCES

1. Clea Parcerisas, Irene T. Roca, Dick Botteldooren, et al. Categorizing Shallow Marine Soundscapes Using Explained Clusters. *Journal of Marine Science and Engineering*, 11(3):550,
March 2023.

2. Carlos M. Duarte, Lucille Chapuis, Shaun P. Collin, et al. The soundscape of the Anthropocene ocean. *Science*, 371(6529):eaba4658, February 2021.

3. Rory Gibb, Ella Browning, Paul Glover-Kapfer, and Kate E. Jones. Emerging opportunities and challenges for passive acoustics in ecological assessment and monitoring. *Methods in Ecology and Evolution*, 10(2):169–185, 2019.

4. Audrey Looby, Christine Erbe, Santiago Bravo, et al. Global inventory of species categorized by known underwater sonifery. *Scientific Data*, 10(1):892, December 2023.

5. John A. Hildebrand. Anthropogenic and natural sources of ambient noise in the ocean. *Marine Ecology Progress Series*, 395:5–20, December 2009.

6. Michelle-Nicole Havlik, Milica Predragovic, and Carlos Manuel Duarte. State of Play in Marine Soundscape Assessments. *Frontiers in Marine Science*, 9, June 2022.

7. Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, et al. The FAIR Guiding Principles for scientific data management and stewardship. *Scientific Data*, 3(1):160018, March 2016.

8. Sourav Mukherjee. Benefits of AWS in Modern Cloud. *arXiv preprint*, March 2019.

9. Shane Guan, Hassan Moustahfid, Anna Milan, and Jacqueline Mize. A Metadata Convention for Passive Acoustic Recordings (Version 1.0), August 2014.

10. Marie A. Roch, Heidi Batchelor, Simone Baumann-Pickering, et al. Management of acoustic metadata for bioacoustics. *Ecological Informatics*, 31:122–136, January 2016.

11. Erik T. Ray. *Learning XML*. O'Reilly Media, 2003.

12. John Worsley and Joshua D. Drake. *Practical PostgreSQL*. O'Reilly Media, Inc., 2002.

13. Léa Bouffaut. *Détection et Classification Dans Un Contexte Acoustique Passive : Application à La Détection Des Signaux Basse-Fréquences Des Baleines Bleues*. PhD thesis, Université de Bretagne occidentale - Brest, October 2019.

14. Clausius D. G. Reis, Linilson Rodrigues Padovese, and Maria C. F. De Oliveira. Automatic detection of vessel signatures in audio recordings with spectral amplitude variation signature. *Methods in Ecology and Evolution*, 10(9):1501–1516, September 2019.

15. Emmanuelle C. Leroy, Karolin Thomisch, Jean-Yves Royer, et al. On the reliability of acoustic annotations and automatic detections of Antarctic blue whale calls under different acoustic conditions. *The Journal of the Acoustical Society of America*, 144(2):740–754, August 2018.

16. Paul Nguyen Hong Duc, Maëlle Torterotot, Flore Samaran, et al. Assessing inter-annotator agreement from collaborative annotation campaign in marine bioacoustics. *Ecological Informatics*, 61:101185, March 2021.

17. Apolline De-Wit, Nicolas David, François Becirspahic, and Frank Fuchs. Etat de l'art des outils existants permettant d'obtenir et saisir les données d'apprentissage, 2021.