dataesr/scientific-paragraphs-categorization huggingface dataset:
Indeed to date, the success of deep neural networks in the supervised regime depends on the availability of a massive corpus of audio examples for the sound events of interest, paired with human annotations. Yet, temporally-precise and fine-grained annotation of audio demands expertise, and is thus costly and time-consuming. In many cases, the obstacle is not only to acquire annotations, but also the audio examples themselves: e. g. for rare species, remote locations, or costly equipment. Furthermore, these numerous small-data scenarios remain outside the scope of digital bioacoustic archives, such as Xeno-Canto and the Macaulay Library. In this article, we aim to guide the development of an unified method that works across the many subdomains of computational bioacoustic sound event detection (SED).
The benefit of doing so resides in the development of a robust and versatile system that could serve the scientific community at large. Hence, we assembled a collection of 14 small-scale datasets, between 10 minutes and 10 hours in duration. Each of them reflects a genuine but slightly different application setting and are obtained from completely different sources. The main originality of our work is in the proposal that, instead of training 14 individual machine listening systems (one per dataset), we train a single system to detect sound events on many different datasets, in which each dataset has a different category of sound event to be detected-that category only defined at "query time". Furthermore, when being evaluated on an audio file, the system is prompted with the first five occurrences of the sound event of interest.
This paradigm of machine learning is known as "few-shot learning"(FSL)  (Snell et al. , 2017; Wang et al. , 2020a). Stated otherwise, our hypothesis is that bioacoustic event detectors can take advantage of whichever bioacoustic datasets are available at training time, and then generalise from a few (five) examples of the new target at deployment time. This is difficult under a standard supervised paradigm because the training set may not reflect real-world deployment conditions, nor cover all sound categories of possible interest. For these reasons, we place the concept of domain adaptation at the heart of the few-shot learning paradigm in bioacoustics: our goal is not only to learn a detector from limited labeled data but also to learn domain-agnostic representations of animal sounds which can readily adapt to unforeseen recording conditions (cf.
Beery et al. (2018) in computer vision). In order to diversify methods and accelerate progress, we have organised an open-science challenge for a community of researchers named DCASE: "Detection and Classification of Acoustic Scenes and Events". The challenge was open to everyone and consisted of public datasets, evaluation metrics, documentation, and baseline systems. In this paper we formulate bioacoustic sound event detection (SED) as a fewshot learning task. We describe our approach in the form of two ML systems customised to this new task (published openly as baseline methods), and we report on a public data challenge conducted over three years to generate and evaluate novel algorithmic solutions. 1  We evaluate various dimensions of the ML paradigms that have been put forward for this task, and explore their ability to adapt to aspects of bioacoustic data presented in our datasets.
Related work Few-shot classification has been investigated generally, and also for audio (acoustic) data  (Snell et al. , 2017; Pons et al. , 2019; Shi et al. , 2020; Naranjo-Alcazar et al. , 2022). However, SED has different requirements from classification: typically, the desired output includes the onset and offset times for each detected event  (Mesaros et al. , 2016) , roughly similar to the "object detection" task in computer vision. One important insight behind few-shot learning is that of meta-learning ("learning to learn"), or the idea of leveraging past experience to speed-up new learning by improving the performance of the learner  (Schaul and Schmidhuber, 2010). One approach to meta-learning is training a system across many loosely-similar tasks/datasets, such that the system is then well-configured to generalise from a few examples of a novel class  (Ravi and Larochelle, 2017; Wang et al.
, 2020a). This depends on a system learning something of the implicit commonalities and analogies across the tasks, which might then influence an algorithm's learnt feature extraction, or its measure of similarity between data points, for example. Related work in computer vision explores the challenge of fine-grained classification and object detection in images from camera traps in novel conditions  (Beery et al. , 2018). Van Horn et al. (  2021 ) introduced a wildlife image dataset with multiple subsets each defining a different binary question. This has a similar meta-learning spirit as our work, with the aim that a sophisticated first stage of "representation learning" across multiple tasks can make future tasks simple. However, unlike the few-shot setting that we use, a lightweight (shallow) classifier must be trained for each new question from a non-trivial number of positive/negative examples.
Novel approaches Deep learning models for few-shot learning problems can be broadly categorized into two approaches: meta-learning and transfer learning. Prototypical networks and matching networks  (Vinyals et al. , 2016) , are good examples of meta-learning that have performed well in few-shot learning tasks across both image and audio domain. Meta-learning based methods rely on the assumption that the tasks belong to a single distribution, for example metric learning based methods require the tasks all coming from a similar domain such that there exists a uniform metric that could work across tasks  (Wang et al. , 2019a). However, in real world scenarios this assumption does not always hold such as in case of our task where the datasets vary in terms of species, recording conditions and microphones, essentially rendering the problem as a cross-domain few-shot learning.
Task formulation We formulate few-shot bioacoustic sound event detection (FSED) as follows: Given one long audio recording (or multiple audio recordings), as well as annotations on the onset and offset time for each of the first five sound events of interest, identify the onset and offset times for all other such sound events in the recording(s) (Figure  1a ). To train a system for this using meta-learning, we make use of multiple bioacoustic datasets (Figure  1b ) representing a range of taxa and recording conditions, each annotated with a different target sound category (see next section). Note that we do not consider multiple classes in one dataset  (Naranjo-Alcazar et al. , 2022; Mesaros et al. , 2019) : each dataset represents a single-class problem. Other sounds are undoubtedly present in almost all audio recordings, but these are considered to be background noise (clutter/distractor events).
Datasets A conventional machine learning experiment uses a single dataset partitioned into three subsets, used for training, validation, and evaluation (test). In the few-shot formulation, we also divide the data into these three partitions (training/validation/evaluation), but each in fact contains multiple datasets, and each dataset represents one example of a few-shot task. Within each dataset, there are one or more audio files, each accompanied by a CSV text file giving the start time, end time and label of the targeted audio events. The label can be POS for a positive example of the target class, NEG for a negative example (background or non-target sound event), or UNK for unknown cases, where the human annotator(s) were unsure whether a sound event should be considered in the positive class.
Such UNK cases may often occur in complex wildlife sound scenes; our chosen strategy was to explicitly label these cases, allowing algorithm designers to make their own decisions on how to handle them, but to exclude UNK time-regions from the evaluation measures (described later) since their correct label is ambiguous. For each dataset, the first five POS events are the "few shots" from which the rest should be inferred. A development set was provided for the task when the challenge was launched, consisting of the predefined training and validation sets to be used for system development. The development set consists of datasets from multiple sources with audio recordings and associated reference annotations in our specified format. More specifically, for the training set multi-class temporal annotations were provided for each recording (with multiple POS/NEG/UNK columns in the data, one per class), while for the validation set single-class temporal annotations (POS/UNK) were provided for each recording.
Jackdaws (JD-Training set): The JD dataset contains a 10-minute onbird sound recording of one male jackdaw during the breeding season in 2015. This individual was recorded in a larger multi-year field study (Max-Planck-Institute for Ornithology, Seewiesen, Germany) in which wild jackdaws were equipped with small backpacks containing miniature voice recorders (Edic Mini Tiny A31, TS-Market Ltd. , Russia) to investigate the vocal behaviour of individuals interacting with their group and behaving freely in their natural environment. Jackdaws are highly vocal corvid songbirds that usually breed, forage and sleep in large groups, but form a pair bond with the same partner for life. The sound recordings contain loud recordings of the focal bird, as well as background sounds from non-focal birds and other sound sources.
Fieldwork was conducted by Lisa Gill, Magdalena Pelayo van Buuren and Magdalena Maier. Sound files were manually annotated by Lisa Gill, using Audacity software, following a previously established video-validation in a captive setting  (Stowell et al. , 2017). Western Mediterranean Wetlands Bird Dataset (WMW-Training set): The WMW dataset contains 161 files with bird sounds from 20 endemic species that are typically inhabitants of the "Aiguamolls de l'Empordà" natural park in Girona, Spain. The audio files that compose this dataset were originally retrieved from the Xeno-Canto portal  (Vellinga and Planqué, 2015). Xeno-Canto is a portal in which citizens can upload wildlife sounds. As the audio files are collected by a wide community of people, the recording devices used for gathering data can be different in every audio file.
Depending on the species, audios contain vocalisations such as bird calls or songs; or sounds such as bill clapping (Ciconia ciconia species) or drumming (Dendrocopos minor species). For the WMW dataset, Juan Gómez-Gómez, Ester Vidaña-Vila and Xavier Sevillano manually cleaned and labelled downloaded audio files using the Audacity software  (Gómez-Gómez et al. , 2023). The cleaning and labelling process consisted in listening to every audio file in the dataset and annotating the specific parts of the file in which the bird is vocalizing, thus separating the bird vocalizations from background noise. HumBug (HB-Validation set): The HB dataset contains sounds of labcultured Culex quinquefasciatus mosquitoes from Oxford, UK, and various species captured in the wild in Thailand, placed into plastic cups  (Li et al.
, 2018). Mosquitoes produce sound both as a by-product of their flight and as a means for communication and mating. Fundamental frequencies vary in the range of 150 to 750 Hz  (Kiskin et al. , 2020). As part of the HumBug project, acoustic data was recorded with a high specification field microphone (Telinga EM-23) coupled with an Olympus LS-14. The recordings used in this challenge are a subset of the datasets marked as 'OxZoology' and 'Thailand' from HumBugDB  (Kiskin et al. , 2021)   2. Polish Baltic Sea bird flight calls (PB-Validation set): The PB dataset consists of six 30-minute recordings of bird flight calls recorded along the Polish Baltic Sea coast (Dąbkowice near Darłowo). Three autonomous recording units were used with the same hardware settings (Song Meters SM2, Wildlife Acoustics, Inc).
They were deployed close to each other (<100m) -near the lake, on the dune, and on the forest clearing -to provide diverse acoustic background. The recordings were acquired during the 2016, 2017 and 2018 fall migration seasons. The recordings are the excerpt from Hanna Pamuła's project, focused on the acoustic monitoring of birds migrating at night along the Polish Baltic Sea coast  (Pamula, 2022; Pamuła, 2022). The passerines night flight calls were annotated by Hanna Pamuła using the Audacity software. In each recording, only one bird species is the target class: song thrush, Turdus philomelos (3 recordings); blackbird, Turdus merula (3 recordings). The usual fundamental frequency range for calls of the chosen species is 5-9 kHz, with standard call duration in the range of 10-250 milliseconds.
Transfer-Exposure-Effects dataset (CHE-Evaluation set): The CHE dataset contains bird vocalizations from the Chornobyl Exclusion Zone (CEZ). Data were collected using unattended acoustic recorders (Songmeter 3) to capture the Chornobyl soundscape and investigate the longterm effects of the nuclear power plant accident on the local ecology. This dataset comes from the Transfer Exposure-Effects (TREE) research project 3. To date, the study has captured approximately 10,000 hours of audio from the CEZ. The fieldwork was designed and undertaken by Mike Wood (University of Salford), Nick Beresford (UK Centre for Ecology & Hydrology) and Sergey Gashchak (Chornobyl Center). Common Chiffchaff (Phylloscopus collybita) and Common Cuckoo (Cuculus canorus) vocalisations were manually annotated and labelled from these recordings by Helen Whitehead.
BIOTOPIA Dawn Chorus (DC-Evaluation set): The DC dataset used as part of the evaluation set stems from dawn chorus recordings, made using Zoom H2 recorders at three different locations in Southern Germany (Haspelmoor, Munich's Nymphenburg Schlosspark, and Nantesbuch). Many bird species produce vocalisations during the entire day, but their vocally most active period by far usually occurs around dawn. This natural phenomenon of dawn chorus has received a lot of attention in biological studies, and also appears to be the perfect time window for species detection, as it provides the largest likelihood of most individuals of the same and of different species signalling. Yet the sheer complexity of undirected dawn chorus recordings have made automatic species classification extremely difficult, leaving this potentially rich source of acoustically determined species data largely untapped.
The Dawn Chorus project is a worldwide citizen science and arts project bringing together amateurs and experts to experience and record the dawn chorus at their doorstep, to draw a global picture of bird biodiversity through sound. The three recordings used in the present study were made and donated by two participants (Moritz Hertel and Rudi Schleich). The vocalisations of three target species (Common cuckoo, Cuculus canorus; European robin, Erithacus rubec-ula; Eurasian wren, Troglodytes troglodytes) were manually annotated by Lisa Gill, using Audacity. Coati (CT-Evaluation set): The CT dataset contains audio recordings collected from two adult females from the same group on Barro Colorado Island, Panama in March 2020. These data are part of the Communication and Coordination Across Scales project.
The two coatis wore collars which recorded high-resolution GPS data with an external attachment of a small audio recording device (TS Market, Edic Mini Tiny+ A77, 22. 05 KHz). Audio data were recorded during their active foraging period in daytime hours when a variety of social and aggressive calls are commonly emitted. Coatis are omnivorous diurnal mammals that live in stable social groups consisting of females and related juvenile and subadult males. Coatis produce a number of call types that are used across a variety of different behavioural contexts. The documentation of their complete vocal repertoire is currently under development. The target calls used in this dataset are growls, chitters and chirp-grunts. Growls and chitters are used in aggressive contexts, whereas chirp-grunts are contact calls emitted when foraging and moving with the group.
Several other call types that might be confused with the targets were captured in the recordings which present the main challenging aspect of this data. Fieldwork was carried out by Emily Grout, Josué Ortega and Ben Hirsch. Calls were labelled by Emily Grout using Adobe Audition. Manx Shearwater (MS-Evaluation set): The MS dataset contains vocalizations from Manx Shearwater individuals, which are procellariform seabirds that breed in dense island colonies in the North Atlantic, mostly in the British Isles, and winter in the South Atlantic off the South American coast. In a multiyear study, Audiomoth recorders were placed in burrows on Skomer Island to record the vocalisations of both adult Manx shearwaters and chicks during the breeding season. Adult Manx shearwaters make loud, distinctive vocalisations while present at their breeding colony in various contexts: in duets with their partner in their nesting burrow, to broadcast from their burrow, and during flight.
Pairs of Manx shearwaters raise single chicks in underground burrows, regularly visiting the breeding colony at night to feed their chick. During these visits, the chick vocalises to 'beg' for food from the parent shearwater; these vocalisations typically comprise bouts of short high-pitched 'peeps'. Fieldwork was undertaken by various members of the Oxford Navigation Group (OxNav), associated with the Oxford University Department of Biology and led by Professor Tim Guilford. Annotation of individual chick begging vocalisations was carried out by Joe Morford using Sonic Visualiser; these vocalisations, therefore, represent the target class in this dataset. Dolphin Quacks (QU-Evaluation set): The QU dataset contains recordings from bottlenose dolphin sounds from Sarasota, FL, obtained using soundand-movement recording DTAGs  Johnson and Tyack (2003a) , attached with suction cups by Frants Jensen in collaboration with Drs.
These calls are produced in bouts often with hundreds of quacks in a single short vocal bout. Bouts of quacks were extracted from 4 bottlenose dolphins tagged in 2013, 2014, and 2015. Quacks were labelled by Austin Dziki and validated by Frants Jensen using DTAG auditing tools in Matlab. Chick calls (MGE-Evaluation set): The MGE dataset contains three 10-minute recordings from three 1-day old domestic chicks (Gallus gallus). Vocalisations have been recorded in 2019 and annotated using Sonic Visualiser in the Prepared Minds Lab (Queen Mary University of London 4  ) by Dr Versace's staff (Shuge Wang, Michael Emmerson, Laura Freeland, Elisabetta Versace). Individual chicks had just been removed from the hatchery, and were free to explore the experimental arena. Chicks have been recorded in the controlled environment of the laboratory, a 24-48 hours after hatching.
Chickens are a precocial social bird species and upon hatching they establish a strong attachment to their social companions, via a process called imprinting, where acoustic information strengthens affiliative responses  Versace et al. (2017). During and after the imprinting process, chicks vocalise signaling that they are in close proximity to their social partners (i. e. pleasure calls) or that they are distant or separated from them (i. e. contact calls). The data gathered in the dataset present uneven time distribution. Calls typically have a short duration (100-400 milliseconds). In the dataset, only pleasure calls were annotated in recordings from chicks one and two, only contact calls were annotated in recordings from chick three. We defined calls based on previous literature  (Marx et al.
, 2001). To summarise, these datasets together represent some of the wide variety of bioacoustic SED tasks, and were selected to give broad coverage of some of the key axes of variation, such as rate of occurrence of the target sound, length of calls, background noise (SNR), taxa, etc. Some of these quantitative characteristics are summarized in Table  1 , and a visual representation of each dataset is presented in figure A. 8. Descriptive analysis of the datasets further illustrates the variation in temporal and spectral characteristics, for the target sounds as well as the background soundscapes. The spectral profile of each dataset is presented in figure  2 , this shows the energy distribution across frequency bands for the POS and background in separate. A similar representation is used to create the temporal profiles shown in Figure  3.
The datasets represent diverse challenges for the few-shot SED systems that are trained and evaluated on them. For each dataset, the provided 5 events are used to specify the class of target sounds. The extent to which a small set of calls can be representative depends on various factors including stereotypy -the degree of how stereotyped are the calls, and vocabulary size. To approximately quantify stereotypy, for each class in the evaluation set, we calculate similarity between sound events. We do this between the selected five events and the remaining events, as well as for the annotated calls more generally (Figure  4 ). Together with the SNR and the sparsity/density of call events, this stereotypy aspect is expected to be one of the axes of variation among our datasets. (details in Appendix A.
Similarity 5shots Stereotypy Figure  4 : Values of similarity between the annotated calls and the first 5 events (shots), and stereotypy for each class in the evaluation set. Classes are indicated in the horizontal axis by DatasetName_ClassName. Both factors are computed using a similarity metric based on the average maximum cross correlation between events. It ranges between 0 and 1, where values closer to 1 represent higher similarity. (the details on how these values are computed are presented in Appendix A. 2).
Evaluation and public challenge For the evaluation of this task, we employ an event-based F-measure with macro-averaged metric, to evaluate the match between true and predicted events. The main complexity is related to the detection of a match between ground truth events and predicted events. Traditional approaches use onset detection based metrics and fixed-size evaluation windows  (Mesaros et al. , 2019). Given the great variation between datasets and characteristics of the events we want to detect in this task, these approaches are not suitable. Instead, we use the Intersection over Union (IoU), with 30% minimum overlap to produce a list of possible matches of the predictions. Applied to temporal events we get a list of predicted events that overlap at least 30% with the ground truth events and thus are candidate matches.
This decision is thus very subjective and dependent on each dataset. For evaluation purposes, matches to UNK events do not count towards calculating TP, FP or FN. In doing so, we ensure that the subjectivity associated with assigning the UNK label does not impact the performamce score of the systems. The procedure we employ is: 1. Apply IoU and bipartite graph matching between predicted events and ground truth POS events only, resulting in TP. 2. Apply IoU and bipartite graph matching between remaining predicted events, that did not match with any POS event, and ground truth UNK events only. 3. Compute FP as the number of predicted events that were not matched to either POS or UNK events. 4. Compute FN as the number of POS ground truth events that were not matched by any predicted event. This is applied to each dataset in the evaluation set where we compute the Fscore metric.