Can Masked Autoencoders Also Listen to Birds?
Authors: Lukas Rauch
René Heinrich
Ilyass Moummad
Alexis Joly
Bernhard Sick
Christoph Scholz
Venue: Accepted by TMLR ()

Abstract: Masked Autoencoders (MAEs) learn rich representations in audio classification through an efficient self-supervised reconstruction task. Yet, general-purpose models struggle in fine-grained audio domains such as bird sound classification, which demands distinguishing subtle inter-species differences under high intra-species variability. We show that bridging this domain gap requires full-pipeline adaptation beyond domain-specific pretraining data. Using BirdSet, a large-scale bioacoustic benchmark, we systematically adapt pretraining, fine-tuning, and frozen feature utilization. Our Bird-MAE sets new state-of-the-art results on BirdSet’s multi-label classification benchmark. Additionally, we introduce the parameter-efficient prototypical probing, which boosts the utility of frozen MAE features by achieving up to 37 mAP points over linear probes and narrowing the gap to fine-tuning in low-resource settings. Bird-MAE also exhibits strong few-shot generalization with prototypical probes on our newly established few-shot benchmark on BirdSet, underscoring the importance of tailored self-supervised learning pipelines for fine-grained audio domains.

Published in Transactions on Machine Learning Research (08/2025)
Can Masked Autoencoders Also Listen to Birds?
Lukas Rauch University of Kossel
Tukas.rauche www.hhhuninssss......... de
René Heinrich Fraunhofer IEE
ren..heiorrcheiie.faamaooo de
Iyass Moummad INRJA Montpellier
ilyuss. Mapss..moummmnllinrrraaj
Alexis Joly INRJA Montpellier
alemis foniisjooyyiinri..ff
Bernhard Sick
b.sick@uni wwwtsnii--ssll.cce de
University of Kassel
Christoph Scholz
daristoph.schokkBeeeeffmbbba
Fraunhofer IEE
Reviewed on OpenReview: https: // openreview. wet/ forum? id= CIBWROXO0
Abstract
Masked Autoencoders (MAEs) learn rich representations in audio classification through an efficient self-supervised reconstruction task. Yet, general-purpose models struggle in fine-grained audio domains such as bird sound classification, which demands distinguishing subtle inter-species differences under high intra-species variability. We show that bridging this domain gap requires full-pipeline adaptation beyond domain-specific pretraining data. Using BirdSet, large-scale bioacoustic benchmark, we systematically adapt pretraining, fine-tuning, and frozen feature utilization. Our Bird-MAE sets new state-of-the-att results on BirdSet's multi-label classification benchmark. Additionally, we introduce the parameter- efficient prototypical probing, which boosts the utility of frozen MAE features by achieving up to 37 mAP points over linear probes and narrowing the gap to fine-tuning in low-resource settings. Bird-MAE also exhibits strong few-shot generalization with prototypical probes on our newly established few-shot benchmark on BirdSet, underscoring the importance of tailored self-supervised learning pipelines for fine-grained audio domains.
224x224
128x
128x512
ImageNer
AudioSet
BirdSet
EXRR
Figure 1: Input modalities. Natural images show local spatial correlations. Audio spectrograms exhibit diverse time- time-ffregeee freeufryy patterns. Bird sound spectrograms reveal sparse, harmonic vocal structures.
1 Introduction
Representation learning through self-supervised learning (SSL) has emerged as a dominant paradigm in audio classification (Huang al., 2022: Chen et al., 2023; 2024a), mirroring its impact in computer vision (He al., 2022; Oquab et al., 2024) and NLP (Touvron al., 2023). By leveraging vast amounts of unlabeled

---

Published in Transactions on Machine Learning Research (08/2025)
data, SSL models learn robust and generalizable representations, often surpassing task-specific supervised models on downstream tasks (Brown et al., 2020). The recent, success of masked image modeling (MIM) (He et al., 2022; Chen et al., 2020a) has established it as one of the prevalent SSL pretraining paradigms in vision (Alkin et al., 2025) and audio (Huang et al., 2022). In particular, masked autoencoders (MAEs) (He et al., 2022) efficiently learn rich representations by reconstructing masked inpuls, making them scalable for pretraining on large datasets (Bao et al., 2022). However, adapting MAEs from vision to general audio requires addressing the structural properties of spectrograms, such as their distinct local redundancies and time- freeufryyy correlations compared to natural images (cf. Figure 1). This motivated the development of the general-domain Audio-MAE (Huang et al., 2022), pretrained on AudioSet (Gemmeke et al., 2017).
While fine-tuned Audio-MAEs demonstrate competitive performance on audio benchmarks beyond AudioSet, such as ESC-50 (Piczak, 2015), their direct transfer to more fine-grained audio tasks is limited. For instance, general-purpose models exhibit a notable performance gap in specialized tasks such as bird sound classífication compared to domain-specific supervised models (Ghani et al., 2023; Hamer et al., 2023). Although AudioSet also contains bird sounds, its coarse-grained nature fails to equip pretrained models with the fine-grained discriminative information required for bird sound classification. In this domain, models must distinguishing between acoustically similar species (low inter-class variation) while handling diverse vocalizations within a single species (high intra-class variation) (Rauch et al., 2024b). Compounding this domain mismatch, MAE's reconstruction objective yields representations that require fine-tuning, offering limited utility as frozen features (Alkin et al., 2025), a drawback common in audio SSL (Chen et al., 2023; 2024a). This presents a critical trade-off: Adapting general-purpos model via full fine-tuning on different tasks with scarcely labeled data is resource-intensivee (Han et al., 2024), while creating a domain-specific foundation model from scratch entails an upfront computational investment. For domains where general audio modcls seem to plateau (Turian et al., 2022), this one-time investment is justified by the benefits unlocked across the rewmstream lifecycle. A domain-specific foundation model enables efficient adaptation to various sub-tasks (e.g., population density or call-type classification in bioacoustics) via lightwiight probing of frozen representations (Ghani et al., 2023). This efficiency is crucial in fields like bioacoustics, where researchers (e.g., biologists) often face scarce labels and have limited computational resources for edge deployment (Bellafkir et al., 2023).
Given these challenges of fine-grained classification and the need for downstream efficiency, bird sound classification emerges as ideal testbed for investigating our core hypothesis: achieving state-of-the-ane (SOTA) performance in audio currently requires a holistic, domain-aware adaptation of the entire SSL pipeline, including pretraining, tie-tuning and frozen feature utilization. Current SOTA models in this area, such as Perch (Hamer et al., 2023), still rely on supervised lcarning, failing to capitalize on the vast unlabeled bioacoustic data suitable for SSL. The large-scale BirdSet benchmark (Rauch et al., 2024b), comparable pretraining data volumes to AudioSet,, provides the necessary resources to test this hypothesis. Thus, we introduce Bird-MAE, a model pretrained exclusively on bird vocalizations via fully adapted SSL pipeline. To better leverage its frozen representations, we propose novel application of prototype learning (Heinrich et al., 2025) as an efficient probing mechanism (i.e., prototypical probing) and systematically evaluate it in low- and high-data regimes. Our key contributions are summarized as follows:
(1) We empirically demonstrate the domain gap between a general-purpose model and a domain- specific model. We emphasize the necessity of domain-speeific SSL for fine-grained audio tasks.
(2) We holistically adapt the MAE pipcline for bird sound classification." a We revisit design choices in the pretraining recipe, fine-tuning methods, and utilization of frozen representations.
(3) We introduce Bird-MAE', a dommain-specific MAE trained on BirdSet. It sets new SOTA on BirdSel, improving mAp by 15 percentage points (pp) on average across tasks over prior best models.
(4) We introduce prototypical probing, a parameter-efficient probe for frozen features. It lifts mAP by up to 37 pp over lincar probes and roduces the fino-tuning gap to 3.3 pp on average across BirdSet.
(5) We establish a few-shot multi-label benchmark for BirdSet, where Bird-MAE with prototypical probing approaches full-data performance, highlighting its efficiency in low-data regimes.
"kttpe://githubb wttp//e/////eeeeeeeeeeeee rewearrrrreeeeeeeeeeeee wew.eeeeagrooeeeeeeeeeeeeeeeeeee MAE
Contributions
2

---

Published in Transactions on Machine Learning Research (08/2025)
2
Related Work
SSL in audio classification. SSL in audio classification has advanced the field, spanning from environmental sounds like ESC-50 (Piczak, 2015) to largo-scale benchmarks like AudioSet that covers wide range of sounds (e.g,, human, animal, musical). Analogous to ImageNet (Deng et al., 2009) in vision, AudioSet provides a large-scale dataset for pretraining SSL models and evaluating their learned representations. While speech SSL models like Wav2vec2 (Baevski et al., 2020) usually operate on waveforms, general audio classification succeeds by adapting vision-based SSL techniques to spectrograms, achicving SOTA results on AudioSet (Chen et al., 2023; 2024a). MIM has successfully transitioned to audio classífication by reconstructing masked spect patches, introducing the Audio-MAB (Huang et al., 2022). This pretraining paradigm offers computational efficiency and fosters the learning of rich audio reprosentations from unlabcled data. Subsequent work, such BEATs (Chen et al., 2023) and EAT (Chen et al., 2024a), further progress audio MIM, incorporating a teacher-student approach. Our work centers on the MAF architecture. Its conceptual simplicity and pretraining efficiency make it an ideal baseline for isolating the impact of domain-specific adaptation and assessing the efficacy of novel probing techniques in challenging fine-grained domains.
Transfer learning in audio classification, General-purpose audio SSL models have proven effective for diverse downstream tasks (Turian et al., 2022; Saeed et al., 2021). BEATss (Chen et al., 2023), EAT (Chen et al., 2024a) and Audio-MAE (Huang et al., 2022) demonstrate their performance on tasks such speech emotion recognition environmental sound classification. However, recent studies reveal notable performance degradation when these general-purpose models are applied to highly specialized domains, particularly bioacousties (Hamer et al., 2023; Ghani et al., 2023). Benchmarks designed for transfer learning, such as HEAR (Turian et al., 2022), and bioacoustic benchmarks like BirdSet (Rauch et al., 2024b) or BIRB (Hamer et al., 2023) highlight this limitation. Specifically, Audio-MAE (Huang et al., 2022) performs worse than spectrogram- based features from supervised models in bioacoustic tasks (Ghani et al., 2023). This performance drop underscores the current. limitations of relying on general-domain pretraining for more fine-grained audio tasks and motivates the development of domain-speciifi solutions. In this work, we address this limitation by introducing and evaluating Bird-MAE specifically adapted to bird sound classification, quantifying the benefils of domain-specific solutions for fine-grained classification in audio.
Downstream task adaptation in SSL. Adapting models to downstream tasks typically involves full fine-tuning or utilizing frozen representations with lightweight probes (Marks et al., 2025). While fine-tuning often yields the highest performance, it can be computationally expensive and may lead to overfitting on smaller datasets. Thus, utilizing frozen representations has gained notable interest in vision (Oquab et al., 2024; El-Nouby et al., 2024; Xie et al., 2022; Assran et al., 2023). Standard approaches involve extracting features (e.g., via global average pooling or the CLS-token) and training a simple classifier, such as linear probing (Oquab et al., 2024), k-NN probing (Zhou et al., 2021; Kakogeorgiou et al., 2022; Lehner et al., 2024), or shallow MLP probing (Dubois et al., 2022; Fuller et al., 2023; Tschannen et al., 2023). However, it is widely observed that representations learned via generative tasks like MTM underperforn with linear probing compared to contrastive methods (Alkin et al., 2025; He et al., 2022; Park et al., 2023). To mitigate this gap in MIM, upstream feature refinement (Alkin et al., 2025) or alternative probing methods have been explored in vision. Notably, attentive probing (El-Nouby et al., 2024; Lee et al., 2019) applies attention mechanism over patch tokens and improves frozen representations with low computational overhead (Yu et al., 2022; Chen et al., 2024b; Darcet et al., 2025). Another probing paradigm involves prototypical networks (Snell et al., 2017; Palanisamy et al., 2024; Tian et al., 2024), which extract class centroids from frozen representations for a similarity-based class assignment without retraining. Despite these ad advancements in vision, current best-performing audio SSL models (Huang et al., 2022; Chen et al., 2023; 2024a) roly on full-model fine-tuning, suggesting their frozen representations offer suboptimal performance or are not properly utilized. In our setting, we evaluate standard parametric probing methods for downstream adaptation using frozen representations, including linear, MLP, and attentive probing. Additionally, we propose and analyze prototypical probing, utilized from prototypical networks in bioacoustics (Heinrich et al., 2025) for a new purpose: as a lightweight, parameter-efficien. probe for frozen MAE representations, effectively utilizing their spatial features.
Bird sound classification. Supervised learning has dominated research in bird sound classification, typically employing convolutional architectures that remain the top performers on bioacoustie benchmarks (Stowell,
3

---

Published in Transactions on Machine Learning Research (08/2025)
2021; Rauch et al., 2024b). For instance, Google's Perch model (Hamer al., 2023) is based on the EfficientNet architecture (Tan & Le, 2019). The feasibility of large-scale supervised training stems from community-driven platforms like Xeno-Canto (XC) (Vellinga & Planqué, 2015), which currently hosts over 850k weakly-labeled bird recordings. Perch and BirdNeXt (Rauch et al., 2024b) derive their training data from XC. However, reliance on manually curated, non-standardized datasets from these platforms has hindered comparison across studies and methods (Rauch et al., 2024b). The introduction of the BirdSet dataset and multi-label bird classification benchmark, which contains volumes of pretraining data comparable to AudioSet, makes this comparison possible for domain-specific SSL. While SSL has shown promise in speech and general audio classification, its evaluation in bioacousties is less mature. Maturel.M-audiio (Robinson et al., 2025) presents the first audio-language model tailored to general bioacoustics, demonstrating competitive zero-shot performance on BirdSet. Existing domain-specific SSL models for bioacoustics, such as BirdAVES (Hagiwara, 2023) and contrastive models (Moummad et al., 2024), have not yet been evaluated under the standardized conditions provided by BirdSet. This work introduces the domain-specific Bird-MAE and comprchensively evaluates it on BirdSet, establishing new SOTA.
3 Model and Training Methodology for DomainnSpeciffiation
This section details the methodological modifications applied to the baseline Audio-MAB architecture and training procedure as part of our holistic domain specification to bird sounds. We organize these modifications into three modules. First, the pretraining module (M1) outlines our changes to the pretraining recipe of the baseline Audio-MAE. Second, we introduce two modules addressing the main downstream adaptation strategies for a pretrained SSL model. The fine-tuning module (M2) involves modifications for the full model training process, while the frozen representations module (M3) investigates the pretrained model as a fixed feature extractor. In the following, we motivate and detail the modifications within each module.
3.1 Pretraining (M1)
Pretraining lays the foundation for effective SSL by learning representations adaptable to downstream tasks. The core of the Audio-MAE baseline is the pretrained encoder ha : X C FxT + #xXWD with parameters a from AudioSet. Here, F represents the number of frequency bins and T the number of time frames of an input spectrogram x E X. The encoder mAPs x to a patch-based feature map h,, where H and W denote the number of non-overlapping patches along the height and width dimensions, and D is the feature dimension per patch. For instance, given an AudioSet spectrogram image with F= 128 and 1020 with a patch size of 16x 16, the encoder ha produces feature map (x) of dimension x 64 D. Our proposed modifications result in an encoder h, pretrained on bird sounds, denoted hg. The key changes from the baseline Audio-MAE pretraining from Huang et al. (2022) are listed in Table 1 with the detailed ablations in Section 5.2. The modifications within this module include:
Model
Dataset
Img
Decoder
Epochs
Masking
Batch
Mean
Std
Mixup
LR
WD
Audio-MAE
AS-2M
1024x 128
Swin
32
0.8
512
-4.2
4.569
2c-4
1e-4
Bird-MAE
XCL-1. 7M
512x
VIT
150
0.75
1024
-7.2
4.43
0.3
2c-4
1o-4
Table 1: Comparison of pretraining parameters of Audio-MAB (baseline) and Bird-MAE (our model).
Data source. The choice of pretraining data influences downstream performance, especially when adapting models from coarse-grained to fine-grained classification tasks. General-purpose datasets like AudioSet encompass broad spectrum of acoustically distinct classes (high inter-class variation), encouraging models to learn discrininative general features. However, fine-grained domains like bird sound classification require distinguishing bet subtle different. species (low inter-class variation) while handling acoustic variability within each species (high intra-class variation) (Rauch et al., 2024b). AudioSet, despite including some animal sounds, does not adequately prepare models for these fino-grained bioacoustic munnces. Therefore, to develop a model tailored for this challenge, we replace AudioSet with domain-specific pretraining data derived from BirdSet (XCL-1. 7M after curation, see Section 4).
4

---

Published in Transactions on Machine Learning Research (08/2025)
Data processing. The raw pretraining dataset from BirdSet contains over 3 million event samples. However, the raw audio collection suffers from redundancy (e.g., multiple events per file, similar background noise) and class imbalance, which can degrade SSL performance (Balestriero et al., 2023). Inspired by the curation process from Oquab et al. (2024), we apply a small selection procodure based on available metadata to reduce redundancy. Specifically, we limit the maximum number of event samples retained per species and recording file. This process results in our curated pretraining dataset XCL-1. 7M, approximately halving the original size of 3.4 million vocalization events in BirdSet. This curated set used train the encoder h3. Further details on the curation process and the results are provided in Section 5 and the Appendix A.
Training paramcters. Optimizing the pretraining recipe can yield substantial performance gains in SSL (Oquab et al., 2024). Thus, we systematically examine the training recipe of the Audio-MAE baseline and identify modifications that yield improvements for bird sound classification through model optimization. These include adjustments to the decoder architectures, increasing the number of training epochs, refining the masking ratio, increasing the batch size, and incorporating mixup augmentation (Zhang et al., 2018) during pretraining. These key changes are summarized in Table 1, with detailed ablation studies in Section 5.
3.2 Fine-tuning (M2)
During downstream adaptation through fine-tuning, the baseline Audio-MAE applies average pooling to the encoder's output fcature map ha (x) to obtain a compact embedding ha (x) E RD.. This embedding is then fed into linear classification head fo : RR + RC (with parameters & and C classes) trained along the encoder to produce logits fa (h,, (x)). This module details modifications applied during this process.
Domain augmentations. To bridge the inherent domain shift between training and test data in BirdSet we utilize domain-specific augmentations while fine-tuning the encoder. While Audio-MAE also employs augmentations, it does not apply strategies tailored to bioacoustics. Our augmentations, informed by results from Rauch et al. (2024b), are designed to simulate common acoustic variations in bird recordings, such as diverse background noises (i.c., noise mixing). varying signal strengths (i.e., gain mixing), and the co- occurrence of multiple vocalizations (i.e., mixup). We supplement these with spectrogram-level augmentations, including frequency and time masking. Further details on the augmentations are provided in Table 9.
Prototypical pooling. Inspired by the performance improvements of the supervised AndioProtoPNet (Hein- rich et al., 2025; Chen et al., 2019; Donnelly et al., 2022) in bioacousties, we introduce a prototypical pooling layer fa. The prototypical layer explicitly pools the spatial structure of the pretrained encoder's patch embeddings h5(x) E H comparable to atlentive pooling (EI-Nouby et al., 2024). For each class o {I....,C) we learn a set of J class-specific prototype vectors {Do.lý=ı with each prototype Pe.j E R., These prototypes are randomly initialized as learnable parameters, distinct from the encoder's weights. We then compute the cosine similarity scores between each prototype P., and every patch embedding (x))),, The resulting similarity scores are then aggregated via max-pooling across all spatial dimensions to obtain the highest similarity score for each prototype of class c:
Po.j hs(x)a
S.jj = max.
h h= 1=1,.. H: w = =0=1,... 1,..., W.
(1)
h.
This yields J similarity scores (B.. Be.J) for each class c. The prototypical layer So then transforms these class-specific similarity scores into logits. Following Heinrich et al. (2025), this transformation is implemented for each class e by a dedicated linear layer, Je RR + E, which takes the J similarity scores for that class as input to produce a single scalar logit Σ.. Each Ie uses weights that are constrained to be non-negative, ensuring that higher similarity to a class prototype contributes positively to the class logit. We adopt the initialization from Heinrich et al. (2025): weights are set to 1 for uniform initial prototype weighting and biases to -2. This yields a near-zero sigmoid probability for instances with no similarity to the prototypes of a class, which is suitable for multi-label classification. The final logit vector for all classes is formed by concatenating these individual class logits. This design ensures that the prediction for each class is based solely on the evidence from its associated prototypes, leveraging local spatial features for robust classification.
BindSet training data consists of local (directed) recordings, contrasting to test data from omnidirectional soundscapes.
5

---

Published in Transactions on Machine Learning Research (08/2025)
To encourage diversity among the learned prototypes within each class and prevent redundancy, the overall training loss incorporates equally weighted orthogonality loss term, adapted from Donnelly et al. (2022).
3.3 Frozen Representations (M3)
Frozen representations offer compulationalyy eflicient alternative to full fine-tuning (Oquab et al., 2024; Touvron et al., 2023). However, MIM representations primarily capture reconstrucioonorientii
patterns rather than discriminative features (Alkin et al., 2025; Oquab et al., 2024). This limits their direct usability, as the lask may dilute critical classification features across reconstructed regions (Walmer al., 2023). While fully fine-tuning the encoder h typically addresses this issue (He et al., 2022; Park al., 2023), can be computationally expensive and unsuitable for tasks with little available labeled data. Thus, this module investigates probing techniques to leverage frozen representations from the pretrained MAE encoder hg.
Prototypical probing. We propose prototypical probing as parameter-effiiient method to leverage frozen features. This involves adapting the prototypical pooling layer f, (as described in M2) to the frozen encoder hg as a lightweight probing head. It a parametric method: The parameters of the prototype vectors {Pej)j=1 for all classes с and the final class-specific linear layers {ge)e= are trained. Similar to attentive pooling, prototypical probing utilizes the full spatial feature map h2(x) E HH preserving local structural information ion. This might be beneficial for bird sounds, as vocalizat ions typically occupy small regions of the spectrogram, where global averaging may dilute this information. Additionally, prototypical probing paramcter-offiiion as the additional trainable parameters only consist of the prototypes J C D and the final linear layer with total of J + C parameters. While this scales linearly with the number of classes and prototypes, its total size remains negligible compared to the encoder. For instance, with the ViT-L encoder (approx. 300M parameters), prototypical probing for the HSN task adds only about 430k parameters (Table 5). Furthermore, it is often smaller than the overhead of attentive probing, which adds approximately 2D + D parameters (EI-Nouby et al., 2024). Prototypical probing retains the low parameter characteristic of linear probing while efficiently exploiting non-linear spatial information crucial for discriminative performance with frozen MAE features.
4 Data and Processing
Dataset. Our experiments utilize BirdSet (Rauch et al., 2024b), a comprehensive benchmark for multi-label bird sound classification (i.e., classifying bird species based on their vocalizations). Unlike AudioSet, where each 10-second sample captures wide array of sounds, bird calls are typically shorter (within 5 seconds, comparable to ESC-50 (Piczak, 2015)) and are confined to narrow frequency bands. BirdSet aggregates the training data from XC (Vellinga & Planqué, 2015), encompassing approximately 520,000 unique recordings
Dataset
Train Recordings
Train Events
Test Segments
# Classes
Pretvoining
Xeno-Canto Large
XCL
528,434
1,724,598
9,735
Dounstream Tasks
High Sierra Nevada
HSN
5,460
17,938
12,000
21
Poudermill Nature
POW
14,911
2,586
4,560
48
Amazon Basin PER
16,802
5,743
15,120
132
Colombia Costa Rica
NES
16,117
4,034
24,480
89
Hawaiian Islands
UHH
3,626
12,978
36,637
27
France and Spain
NBP
24,327
76,438
563
51
Sapsucker Woods
SSW
28,403
4,285
205,200
81
Sierra Nevada
SNE
19,390
2,557
23,756
56
Table 2: Dataset overview of BirdSet for pretraining and downstream tasks. |Train| Recordings contains the number of recordings, [Train| Segments is the number of extracted samples per task in experiments, and |Test| is the number of 5-second segments. HSN is used for validation and ablations.

---

Published in Transactions on Machine Learning Research (08/2025)
(weakly labeled at the file level) from nearly 10,000 bird species, totaling over million vocalization events. For evaluation, BirdSet provides eight downstream tasks, each consisting of a dedicated training subset and a test set derived from fully annotated soundscape recordings from different geographical regions (e.g., High Sierras Nevada (HSN) or Amazon Basin (PER)). The test sets are segmented into 5-second intervals, where each interval receives multi-label annotations indicating the presence (one or multiple) or absence of birds. This structure explicitly captures challenges like domain shift between training (focal) and test (soundscape) data, as detailed in Rauch et al. (2024b). Table 2 provides a detailed overview of the datasets.
Processing and evaluation. Audio segments standardized to 5 seconds and sampled to 32 kHz. We extract 128-dimensional log-mel filterbank features, following common practice in audio SSL (Huang et al., 2022; Chen et al., 2024a). The resulting input dimensions are fixed at 128x x 512. All results in ablations and benchmark studies are averaged over three repetitions. We report the class-based mean average precision (mAP). Since BirdSet provides no typical validation split per downstream task for dedicated training, we repurpose the HSN downstream task (with only 21 classes) as our development set for hyperparameter tuning and ablation studies. After finalizing all design choices, we retrain the model once on each downstream lask's training data and report performance their test sets.
5 Ablation Studies
This section presents ablation studies to validate the design choices and quantify the impact of each modification module (M1, M2, M3) introduced in Section 3. We analyze these components by sequentially applying them to a bascline Audio-MAE configuration, which uses the original implementation from Huang et al. (2022) detailed in Table 1. We illustrate the cumulative improvements with full fine-luning in Figure 3.
Settings. For modifications related to the pretraining (MI, Section 5.1) and fine-tuning (M2, Section 5.2) modules, we evaluate performance via full model fine-tuning. For frozen representations (M3, Section 5.3), we ablate the effectiveness of different probing techniques (linear, MLP, altentive), compared to our protolypical probing when using the best- pes--performin settings from MI and M2. For each experiment, we report the average over three random scods to account for variability training. All ablation experiments are performed on the HSN multi-label downstream task from BirdSet. Further experimental details and hyperparameters are provided in the Appendix D.
5.1 Pretraining (M1)
Data source. Figure 3 shows that replacing the AudioSet pretraining data with BirdSet for the base model yields a modest performance gain of 2.45 pp in fine-tuning. Even if extensive fine-tuning on large downstream datasets can mitigate pretraining domain mismatch, domain-spocific pretraining data provides a clear advantage, especially for the performance of probing techniques (see Section 5.3). However, the most decisive gains emerge from a holistic adaptation of the entire SST. pipeline to the target domain. Thus, swapping in domain-specifii data is necessary but insufficient: aligning both objective and downstream training with the structure of bird vocalizations in the domain unlocks most of the benefit.
Data processing. The quality and size of the pretraining dataset are crucial for SSL. To quantify the impact of dataset size, we begin with the full XCL-3. 4Mg training dataset with all available sound events and progressively reduce it to 50%, 25%, and 12.5% by random sampling. As shown in Table 3a, pretraining performance generally scales with dataset size when using randomly sampled subsets, although gains diminish beyond 1.7 million samples in our use casc. Noticcably, applying our data curation strategy (balancing classes, reducing redundancy via metadata) to create the curated 1.7 million sample dataset (XCL-1.7M) 7M) results in better performance compared to using an uncurated dataset of the same size or even the full, uncurated 3.4 million sample dataset. This highlights the benefit of data curation, outweighing data volume in this setting. More details of the curation process are available in the Appendix A.
Pretraining recipe. Optimizing the pretraining recipe beyond just the data source further enhances performance. As summarized in Figure 3, modifications like increasod epochs, adjusted masking ratio, larger batch size, and mixup (see Table 3b) sequentially improve downstream results. While changing the decoder architecture did not yield direct performance gains in isolation, it improved training stability. Figure 2
7

---

Published in Transactions on Machine Learning Research (08/2025)
Baseline
23.71
+31.54
Dataset
Selection MAP
+19.93
28.35
XCL-3.4M 4M,
Random
52.18
Mix
MAP
Pool
MAP
Bird Augs. + 43.64
Audio-MAR
XCL-117M,
Random
52.11
0.0
53 63
cls
52.89
XCL-0.8% 84, Random
43.28
XCL-0. 4M,
Random
35.98
0.3
55.28
avg
53.15
VTT-B BirdSet + 46.09
0.5
52.56
atten
53.36
22.75
XCL-1.7M Curated 55.28
0.7
52.39
proto
55.28
VTT-I, BirdSet +
48.84
(a) Dataset size
(b) Mixup
(c) Pooling
Epochs 150
51.02
Table 3: Detailed ablations on (a) dataset size and cura-
Pretraining
tion in SSL, (b) mixup in SSL and (c) pooling in fine-tuning.
Masking 0.75
51.61
+0 50
Batch 1024
52.41
65
-0.10
VIT Decoder +
52.25
30
Fine- Tuning
Mixup 0L.
53.15
MAP
PT Pooling +
40
BAR--MAE-B 52.06
35
VIT-B/16
Bird MAE
55.28
VIT-L/16
VIT-H/16
15
30
45
47
49
51
53
55
5
10
60
120
140
180
200
Epochs
Figure 3: Ablations for improving the base Audio-MAE, The MAP results are reported on HSN. The + symbol indicates new component, while and denote an increase and decrease in a parameter.
Figure 2: Model size and training epochs comparison on HSN. We report the MAP score different pretraining checkpoints in all model sizes.
confirms the benefit of extended pretraining, showing mAP improvements across different VIT sizes up to approximately 150 epochs, after which gains saturate.
5.2 Fine-tuning (M2)
Domain augmentations. Adapting the fine-tuning process with domain-specifie data augmentations is crucial. Our sequential ablation in Figure shows that applying the baseline Audio-MAE to HSN without domain-adapted angmentations yields inferior results (23.71%). Introducing domain-specific augmentations (detailed in the Appendix D) during fine-tuning provides a substantial performance uplift, increasing mAP by circa 29 pp over the baseline implementation. This highlights the importance of domain-aware adaptations, ensuring models can deal with the challenges the domain data (e.g., domain shift BirdSet).
Prototypical pooling- Replacing global averaging the CLS token with prototypical pooling further enhances classification performance when fully fine-tuning the model, setting Bird-MAE-L model ou HSN, this final modification elevates the mAP score to 55.28% from 53.15% when using global average pooling. Prototypical pooling also outperforms alternative advanced pooling mechanisms such as attentive pooling (see Table 3c). This improvement contributes to more than doubling the performance of the initial Audio-MAE baseline (23.71% mAP, before any domain specifications). As shown Table 5, the parameter overhead of this prototypical pooling operation minimal compared the ViT encoder.
SOTA results. For the
5.3 Frozen Representations (M3)
Prototypical probing. We ablate the quality of frozen representations using various probing methods Table We use the best-performing settings from MI and M2, including all augmentations. All experiments use J=20 prototypes. The impact of varying is shown in Appendix F. Consistent with findings MIM (Alkin et al., 2025), results confirm that standard probing techniques applied to features extracted
8

---

Published in Transactions on Machine Learning Research (08/2025)
via global average pooling (linear, MLP) perform poorly with frozen MAE representations, even with the domain-specilic Bird-MAF. mAP scores remain notably lower than full fine-tuning. However, methods explicitly leveraging the spatial feature map achieve considerable performance gains. Attentive probing notably improves results across model sizes. Our prototypical probing further boosts performance, outperforming allentive probing across all Bird-MAE model sizes while remaining more parameter-efficent (Table 5) with approximately 20% of parameters compared to attentive probing for this dataset. Interestingly, prototypical probing performs worse than attentive probing when applied to the general-purpose Audio-MAE, highlighting that prototype- based methods might benefit from domain-specife pretraining. For the Bird-MAE-L model, prototypical probing achieves mAP of 49.97%, a substantial gain from MLP probing (+33.75 pp) and only 5.31 pp below the full fine-tuning result (55.28%). This demonstrates that by effectively utilizing the spatial information preserved in the frozen MAE feature map, prototypical probing addresses the limitations of frozen MIM representations for discriminative tasks. It offers an efficient alternative to full fine-tuning.
VIT
Technique
Linear
MLP
Attentive
Prototypical
Probing
Parameters
HSN
F Fine-tur inn-tuning:
43.11
B/16
Audio-MAN
9.35
10.45
31.43
20.89
VIT-L parameters: 307M
Fine-tuning: 52.06
Linear
C(D+1)
21k
B/16
Bird-MAE
13.06
17.23
43.12
43.84
NLP
1[D+1)+C((H)
535k
Fine-tuning: 55.28
L/16
Bird-MAE
13.29
15.22
47.81
49.97
Attentive
2D (C) L)D C
2.1M
Fine-tuning: 54.05
H/16
Bird-MAE
13.83
18.71
45.73
47.52
Prototypical C. 1) 11
Table 4: Frozen representation ablations on HSN, evaluated with probing techniques. Linear and MLP utilize the global average, attentive and prototypical (J 20) the feature map.
Table 5: Parameters for probing with
example values of HSN: D = 1024, C 21, H 512, and J =0.
6 Benchmark Results
This section presents the empirical evaluation of our domain-specific Bird-MAE model on the BirdSet downstream lasks, comprising multi-label classification of bird species vocalizations. We assess performance under two conditions: BirdSet's multi-label classification using all available training data and our novel few-shot multi-label probing benchmark with limited labeled examples. Our evaluation aims to (1) showcase the importance of a domain-specifie SSL in audio, (2) validate the effectiveness of prototypical probing for leveraging frozen representations, and (3) establish new SOTA results on BirdSet.
Baselines and evaluation. We compare Bird-MAE against several relevant models. Our baseline is the Audio-MAE-Basee pretrained AudioSet and fine-tuned with prototypical pooling. First, compare against other bird-specific SST models: BirdAVES (Hagiwara, 2023) and a SimCTR (Chen et al., 2020b) implementation from Moummad et al. (2024), both pretrained on custom XC data. Second, we include results from the best-performing supervised models in bird sound classification: Google's Perch (Hamer el al., 2023) and BirdSet's BirdNeXt (Rauch et al., 2024b), also pretrained on XC data. We report Bird-MAE results using ViT-Base, ViT-Large, and ViT-Huge backbones, incorporating all modifications from Section Our domain-specific augmentation pipeline is applied during fine-tuning and probing to all models where feasible to ensure fair comparison in both tasks. We exclude spectrogram-level augmentations for the waveform-based BirdAVES model. For Perch3 and BirdNeXt, mask logits for classes not present in the downstream task's label set (Rauch et al., 2024b). Hyperparameters for all models are tuned once on HSN before final evaluation across downstream tasks. More details and hyperparaneters can be found in Appendix D.
Laarger model checkpoints are not available from the source paper (Huang et al., 2022).
Seerch not publicly available for fine-taning.
9

---

Published in Transactions on Machine Learning Research (08/2025)
Model
Arch.
Pretraining
HSN
POW
PER
NES
UHH
NEP
SSW
SNE
Supervised (with masked inference)
Perch NeXt
EffNet-B ConvNeXt
Xeno-Canto"
41.84
30.41 vnl
18.23
38.53
26.71
62.62
28,11
28.45
XCL. BirdSet
47.29
35.63
18.52
33.68
26.07
62.24
35.19
29.88
Self-Superuised (with finettuinng)
BirdAVES
HuBERT CvT-13
Xeno-Canto'
42.63
11.33
5.18
5.72
21.97
70.09
4.45
7.08
SimCLR
Xeno-Canto'
39.33
34.84
16.85
23.54
20.72
65.69
18.67
15.99
Audio-MAE
ViT-B/16
AS-2N
44.69
39.03
21.32
29.83
26.43
67.02
26.94
22.44
ViT-B/16
XCL-1.
52.06
45.24
27.58
37.12
28.48
62.86
32,83
28.04
Bird-MAE
ViT-L/16
XCL-1.7M 7M
55.28
55.26
34.64
41.50
30.17
71.69
40.82
33.82
VIT-H1/16
XCL-1.7M
54.80
54.05
33.29
39.28
29.81
69.35
41.32
32.18
Table 6: Finc-tuning results the multi-label classification benchmark with full data (mAP%). Comparison of SL and SSL models, following the evaluation protocol of BirdSet. Best and second best. results are highlighted. Xeno-Canto" denotes pretraining on unspecified subsets of XC data.
6.1 Multi-Label Classification
Settings. In this section, we first evaluate the performance of the pretrained Bird-MAE on BirdSet's multi-label classification benchmark with full training data. We present the fine-tuning results in Table 6 and the frozen representation results in Table For each experiment, we report the mcan over three randomly initialized runs. More detailed results are available in Section E.1.
What is the performance gain of domain-specific MAE? Table 6 confirms results from ablation studies: Domain specification via Bird-MAE yields substantial performance improvements across the BirdSet benchmark compared to the general-purpose Audio-MAE. While the Bird-MAF-B also offers notable gains over the available Audio-MAE baseline, the benefits become more pronounced with larger architectures. For instance, our Bird-MAE-L achieves notably higher mAP scores than the baseline, with performance gains of +15 pp on POW (+7 pp Bir--MAE-B) or +13 pp on PER (+6 pp Bird-MAE-B). Furthermore, Bird-MAE consistently and notably outperforms the other domain-specific SST baselines AVES and SimCLR across all datascts, often by margins exccoding 15-20 pp mAP on average.
How does the model compare to supervised models? We compare Bird-MAE against the current best-performing supervised models (Perch and BirdNeXt). Our fine-tuned Bird-MAE models consistently achiove new SOTA results across the BirdSet benchmark. Specifically, Bird-MAE-L outperforms the BirdNeXt and Perch baselines on all eight. datasets, often by considerable margins. For example, on SSW, Bird-MAF-L. achieves 40.82% mAP compared to Perch's 28.11% mAP (+12.7 pp), and on PER, it achieves 34.64% mAP versus Perch's 's 18.23% mAP (+16.4 pp). These results demonstrate the effectiveness of domain-specific SSL combined with modern transformer architectures and higher parameter counts compared to prior supervised CNN-based approaches in bird sound classification.
Can we freeze the representations? While audio SSL currently relies on fine-tuning (Chen et al., 2023; 2024a), efficient deployment is highly desirable for edge applications in bioacoustics (Höchst et al., 2022). We evaluate the performance of frozen representations using linear versus our proposed prototypical probing Table Linear probing performs poorly for Audio-MAB and Bird-MAE across backbone sizes, confirming the difficulty of using frozen MIM representations directly. However, prototypical probing drastically improves performance by leveraging the spatial feature map: it closes the gap to fine-tuning to approximately 3 pp mAP on average across downstream tasks. Bird-MAE with prototypical probing also achieves substantial gains over Audio-MAE with prototypical probing (e.g., +27.3 pp base performance on NBP). Additionally, it notably outperforms other SSL models (AVES, SimCLR) using either probing method and also surpasses the fully supervised Perch model (from Table 6) on nearly all datasets using only frozen representations. This challenges the notion that MAE features are unsuitable for probing and demonstrates prototypical probing highly effective and efficient alternative to fine-tuning in bird sound classification. While prototypical
10

---

Published in Transactions on Machine Learning Research (08/2025)
probing enhances performance for the masking-basee BirdAVES model, it seems degrade performance for the contrastive SLR model, suggesting probing effectiveness interacts with the SSL pretraining objective.
Model
Arch.
Probing
ya
POW
PER
NES
UHH
NBP
SSW
SNE
Self-Supervised (with frozen representations)
linear
14.91
12.60
5.41
6.36
11.76
33.68
4.55
7.86
BirdAVES
HuBERT
proto
32.52
19.98
5.14
11.87
15.41
39.85
7.71
9.59
linear
17.29
17.89
6.66
10.64
7.43
26.35
6.99
8.92
SimCLR
CvT-13
proto
18.00
17.02
3.37
7.91
7.08
26.60
5.36
8.83
linear
8.77
10.36
3.72
4.48
10.78
24.70
2.50
5.60
Audio-MAE
ViT-B/16
proto
19.42
19.58
9.34
15.53
16.84
35.32
8.81
12.34
linear
13.06
14.28
5.63
8.16
14.75
34.57
5.59
8.16
ViT-B/16
proto
43.84
37.67
20.72
28.11
26.46
62.68
22.69
22.16
linear
12.44
16.20
6.63
8.31
15.41
41.91
5.75
7.94
Bird-MAE
VIT-L/16
proto
49.97
51.73
31.38
37.80
29.97
69.50
37.74
29.96
linear
13:25
14.82
7.29
7.93
12.99
38.71
5.60
7.84
VYT-H/16
proto
47.52
49.65
30.43
35.85
28.91
69.13
35.83
28.31
Table 7: Probing results on the multi-label classification benchmark with full data (mAP%).
Comparison of linear probing vs. prototypical probing using frozen encoder representations. Models follow the evaluation protocol of BirdSet. Best and second best results are highlighted.
6.2 Few-Shot Multi-Laber Probing
Settings. this section, we evaluate the few-shot learning capabilities of Bird-MAF, introducing a few-shot. multi-label classification benchmark BirdSet to test the frozen representations in a low data regime. This setup maintains the standard test and domain shifts but restricts the training data k E {1,5, 10) event instances per class for each downstream task's 's training subset. We only report our best-performing Bird-MAE-L model with an average of three repetitions and three randomly sampled subsets per shot. Detailed results are available in Appendix E.2 and the few-shot sampling strategy is described in Appendix A.
Linear
Attentive
Protetypical
Prototypical full
HSW
40
40
300
20
MAP
20
20
10
0-
UHH
NBP
SSW
SNE
40
30
30
60
20
40
MAP
10
10
10
k-Shots
k-Shots
k-Shots
k-Shots
Figure 4: Few-shot probing results. We compare linear, attentive and prototypical probes using frozen Bird-MAF-L features at (1,5, 10) shots per class. Results averaged over three three subsets per shot with the standard deviation. The dashed line marks the upper probing bound on the full dataset.
11

---

Published in Transactions on Machine Learning Research (08/2025)
How does few-shot prototypical probing compare to other methods? Figure shows that the advantages of prototypical probing are even more pronounced in low-data regimes. While linear probing yields very low mAP scores across all k-shot settings, prototypical probing delivers substantially better performance, even with just one shot per class on the file level. Attentive probing also provides a clear improvement over linear probing but consistently underperforms compared to prototypical probing across all shot counts. These trends are observed uniformly across all eight datasets, underscoring the effectiveness of prototypical probing in leveraging MAE embeddings for few-shot learning.
Can few-shot prototypical probing rival full dataset probing? Figure 4 demonstrates that prototypical probing with only 10 shots per class achieves performance close to full-data probing. For instance, on the PER, 10-shot prototypical probing reaches approximately 29.31% mAP, approaching the 29.97% mAP of full-data prototypical probing and the 34.64% mAP of full fine-tuning. While attentive probing also shows data efficiency in few-shot scttings compared to linear probing, it generally does not reach the same level of proximity to full-datasct performance as prototypical probing. Comparable trends appear across all datascts, illustrating the data efficiency by combining Bird-MAE's features with prototypical probing.
7 Discussion and Limitations
Several research directions emerge from our findings, framed by the scope and limitations of this study. In the following, we discuss these limitations and show important directions for future work.
Upfront cost of domain-specific pretraining. We advocate for a domain-specific foundation model in complex fields like bioacoustics. This approach requires notable onc-time investment for pretraining, which we frame a necessary trade-off for achieving SOTA performance in challenging fine-grained domains where general-purpose models currently fall short (Hamer et al., 2023; Turian et al., 2022). This initial cost is balanced by a reusable asset for the research community that enables efficient downstream adaptation (Ghani et al., 2023): Various sub-tasks can be tackled by simply training a lightweight probe, avoiding the need for repeated, costly fine-tuning. For instance, Bird-MAF, could be deployed for other bioacoustic tasks like fine-grained call-type classification (Kahl al., 2021) or population density estimation (Navine et al., 2024). Furthermore, our preliminary results in Table 8 suggest this benefit may even possess cross-species transferability within bioacoustics, as Bird-MAE outperforms the general-purpose model the MeerKAT dataset. Future work could focus on making this domain-specific pretraining more computationally efficient.
Model
Arch.
Head
Probing
Model
Head
Probing
Fine-tuning
Audio-MAE
ViT-B
Linear
16.16
Audio-MAE
Linear
13.19
37.3
Audio-MAE
VIT-B
Proto
28.30
Audio-MAE
Proto
17.97
38.6
Bird-MAE
VIT-B
Linear
17.38
BEATs
Linear
38.3
Bird-MAE
VIT-B
Proto
34.58
(a) AS-20k
(b) MeerKAT
Table 8: Generalizability of prototypical layers and Bird-MAE (mAP%). Performance comparison two bird datasets: general audio (AS-20k) and another fine-grained bioacoustic task (MeerKAT). We evaluate the impact of using our prototypical head versus standard linear head for both frozen feature probing and full fine-tuning.
Recipe and results transferability. While our pretraining recipe (e.g., extended epochs, mixup) and prototypical probing are holistically adapted for bird bioacoustics, we do not evaluate their transferability to general audio tasks. To investigate the broader applicability of prototypical probing, we present preliminary experiments on two non-bird datascts: the general audio benchmark AS-20t and another fino-grained bioacoustic task, the MeerKAT mammal vocalization dataset. Details on the MеerKAT dataset are provided in Appendix G. Table provides initial evidence of generalizability. On general audio (AS-20k), simply replacing the standard linear head with our prototypical pooling layer notably improves the performance of the off- thel AAd for both frozen feature probing and full fine-tuning. This fine-tuning result surpasses even the more advanced BEATss (Chen et al., 2023). On the related bioacoustic MeerKAT task,
12

---

Published in Transactions on Machine Learning Research (08/2025)
observe two key results: First, prototypical probing again substantially outperforms linear probing for both the general Audio-MAR. (+12.1 pp) and our Bird-MAR (+17.2 pp). Second, Bird-MAR outperforms the general Audio-MAE with both probes, supporting findings from Ghani et al. (2023) that bioacoustic pretraining can offer bencfits across related fine-grained animal vocalization tasks. These results suggest that our prototype-based methods are not limited to bird sound classification and can unlock greater performance from existing SSL. backbones. comprehensive cross-domain study and extending these methods to other fine-grained settings such as insect call or speaker dialect classification remain important next steps.
Shortfall of general-purpose MAEs. While our study demon- strates that domain-specific and holistically adapted MAE pipeline can excel, there could be other reasons for the performance gap. Our work leaves open questions regarding other factors, such as (1) the model scale or the (2) the SSL objective. Regarding model scale, one might hypothesize that simply using a vastly larger general- model would suffice. However, evidence suggests this approach has limitations. We observe diminishing performance returns when scal- ing our own adapted model from a ViT-Large to a ViT-Huge back- bone. This aligns with findings from benchmarks like HEAR (Turian et al., 2022), where even billion-parameter general audio models have been shown to underperform on specific bioacoustic tasks. Regarding the SSL objective, another hypothesis is that the MAE reconstruc- tion task is inherently less suited for this fine-grained domaín than other methods. Evidence suggests the challenge is broader than one specific method. Our study shows that a holistically adapted Bird- MAE notably outperforms other domain- domain-adapted SSL models that use different objectives, including the contrastive SimCLR and the masking- based AVES Table 6. This aligns with findings from BIRB bioacoustic benchmarks (Hamer et al., 2023), where the YamNET model (weak clip-level training) also exhibits performance limita- tions. Collectively, this suggests that while model scale and SSL objective influence feature quality, comprehensive, domain-specifie adaptation of the entire pipeline appears to be a dominant factor for achieving SOTA performance in this fine-grained domain. Nevertheless, a systematic cross- poos-par comparison on a standardized benchmark like BirdSet remains a valuable direction for future work to definitively isolate the impact of different SST. objectives and model sizes.
Spectrogram
Frequimey
Volume
Heatmap
Frequncy
Sumilariy
Time
Figure 5: Activation heatmap of a prototype superimposed on spectro- gram (Heinrich et al., 2025).
Prototype interpretability. The interpretability of the learned prototypes by heat map visualizations provide notable value beyond performance metrics. Heinrich et al. conduct comprehensive analysis demonstrating that prototypes can learn to represent human-interpretable sound patterns, such as distinct. call types within single bird species (as we illustrate exemplary Figure 5). While detailed interpretability analysis of our specific Bird-MAE representations is beyond the scope of this performance-focuse represents an important direction for future research. For instance, the ability to inspect what a model has learned offers opportunitics for human-in-the-loop learning, where ccologists could validate or refine prototypes to correct model errors and potentially further enhance performance in data-scarce scenarios (Rauch et al., 2024a). This synergy between strong few-shot performance and interpretability could enable the development. of robust and transparent monitoring of of rare species with minimal labels.
study, it
8 Conclusion
In this work, we addressed the limitations of general-purpose SSL models in audio classification. We demonstrated the efficacy of a holistically adapted, domain-specific masked image modeling pipeline for bird sound classification: We revised the entire training process, including pretraining (e.g., replacing AudioSet with BirdSet), finc-tuning (c.g., adding prototypical pooling), frozen representations (c.g., utilizing prototypical probing), leading to the development of our Bird-MAE model. Bird-MAE achieves novel trate-of-tho-at performance the BirdSet multi-label classification benchmark, strongly outperforming the general-purpost
13

---

Published in Transactions on Machine Learning Research (08/2025)
Audio-MAB and prior best- rest-performing supervised models. Our findings highlight that while domain-specific pretraining is crucial, the full benefits of such adaptations become particularly evident when leveraging frozen representations. Specifically, our parameter-efficiatt prototypical probing substantially narrows the gap to full fino-tuning to 3 pp mAP on average across downstream tasks and boosts up to 37 pp over linear probing. These results underscore the importance of domain-aware pretrained features and effective probing methods. Furthermore, Bird-MAE with prototypical probing delivers strong few-shot performance, offering an efficient. alternative for resource-constrained bioacoustic applications. Our study shows that achieving optimal results in more fine-grat fineggriindd audio tasks such bioacoustics requires moving beyond generic SSL approaches towards holistic, domain-aware pipeline adaptations.
14

---

Published in Transactions on Machine Learning Research (08/2025)
References
Benedikt Alkin, Lukas Miklautz, Sepp Hochreiter, and Johannes Brandstetter. Mim-refiner: A contrastive learning boost from intermediate pre-trained representations. In International Conference on Learning Representations (ICLR), 2025.
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. TEEE/CY /CVF Conference on Computer Vision and Pattern Recognation (CVPR), 2023.
Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli. Wav2vec 2.0: A framework for self-supervised learning of speech representations. In Advances Nearal Information Processing Systems (NeurIPS), 2020.
Randall Balestriero, Mark Tbrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, and Yuandong Tian. A cookbook of self-supervised learning. arXi::234.112210, 2023.
Hangbo Bao, Li Dong, and Furu Wei. Beit: Bert pre-training of image transformers. In International Conference on Learning Represenlations (ICLR), 2022.
Hicham Bellafkir, Markus Vogelbacher, Daniel Schneider, Markus Mühling, Nikolaus Korfhage, and Bernd Freisleben. Edge-Based Bird Species Recognition via Active Learning. Networked Systems, 2023.
Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Choss, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, llya Sutskever, and Dario Amodei. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Chaofan Chen, Oscar Li, Danicl Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learning for interpretable image recognition. Advances in Newral Information Processing Systems (NeurIPS), 2019.
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Πya Sutskever. Generative pretraining from pixels. In International Conference on Machine Learning (ICML), 2020a.
Sanyuan Chen, Yu Wu, Chengyi Wang, Shujie Liu, Daniel Tompkins, Zhuo Chen, Wanxiang Che, Xiangzhan Yu, and Furu Wei. Beats: Audio pre-training with acoustic tokenizers. In International Conference on Machine Learning (ICML), 2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. A simple framework for contrastive learning of visual represontations. In International Conference on Machine Learning (ICML), 2020b.
Wenxi Chen, Yuzhe Liang, Ziyang Ma, Zhisheng Zheng, and Xie Chen. Eat: Self-supervised pre-training with efficient audio transformer. In International Joint Conference on Artificial Intelligence (IJCAT), 2024a.
Xiaokang Chen, Mingyu Ding, Xiaodi Wang, Ying Xin, Shentong Mo, Yunhao Wang, Shumin Han, Ping Luo, Gang Zeng, and Jingdong Wang. Context autocncoder for self-supervisod representation learning. International Journal of Computer Vision, 2024b.
Timothée Darcet, Federico Baldassarre, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Cluster and predict latent patches for improved masked image modeling. 2XXi:2502.087639 2025.
Jia Deng, R. Socher, Li Fei-Fei, Wei Dong, Kai Li, and Li-Jia Li. Imagenet: A large-scale hicrarchical image database. In IEEE Conference on Computer Vision and Pottern Recognition (CVPR), 2009.
Jon Donnelly, Alina Jade Barnett, and Chaofan Chen. Deformable protopnct: An interpretable image classifier using deformable prototypes. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
15

---

Published in Transactions on Machine Learning Research (08/2025)
Yann Dubois, Stefano Ermon, Tatsunori B Hashimoto, and Percy S Liang. Improving self-supervised learning by characterizing idealized representations. Advances in Neural Informalion Processing Systems (NewrIPS), 2022.
Alaacldin El-Nouby, Michal Klein, Shuangfei Zhai, Miguel Angel Bautista, Vaishaal Shankar, Alexander Toshev, Joshua M Susskind, and Armand Joulin. Scalable pre-training of large auloregressive image models. In International Conference on Machine Learning (ICML), 2024.
William Falcon and The PyTorch Lightning team. Lightning-AI//A A/Liightnin..
PyTorch Lightning, 2019. URL https:///iihhbb.com
Anthony Fuller, Koreen Millard, and James Green. Croma: Remole sensing representations with contrastive radar-optical masked autoencoders. Advonces Neural Information Processing Systems (NeurIPS), 2023.
Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freodman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and hunan-labeled dataset for audio events. In IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017.
Burooj Ghani, Tom Denton, Stefan Kahl, and Holger Klinck. Global birdsong embeddings enable superior transfer learning for bioacoustic classification. Scientific Reports, 2023.
Masato Hagiwara. Aves: Animal vocalization encoder based self-supervision. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023.
Jenny Hamer, Eleni Triantafillou, Bart Van Merriënboer, Stefan Kahl, Holger Klinck, Tom Denton, and Vincent Dumoulin. Birb: A generalization benchmark for information retrieval in bioacousties. 074:233920749, 2023.
Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprchensive survey. Transactions on Machine Learning Research (TMLR), 2024.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollar, and Ross Girshick. Masked autoencoders are scalable vision learners. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
René Heinrich, Lukas Rauch, Bernhard Sick, and Christoph Scholz, Audioprotopnet: An interpretable deep learning model for bird sound classification. Ecological Informatics, 2025.
Ya-Yao Huang, Hu Xu, Juncheng Li, Alexei Baevski, Michael Auli, Wojciech Galuba, Florian Metze, and Christoph Feichtenhofer. Masked autoencoders that listen. Advances in Neural Information Processing Systems (NeurIPS), 2022.
Jonas Höchst, Hicham Bellafkir, Patrick Lampe, Markus Vogelbacher, Markus Mühling, Daniel Schneider, Kim Lindner, Sascha Rösner, Dana G. Schabo, Nina Farwig, and Bernd Freisleben. Bird@edge: Bird species recognition at the edge. In Networked Systems. 2022.
Iver Jordal, Shahul ES, Hervé Bredin, Kento Nishi, Francis Lata, Harry Coultas Blum, Pariente Manuel, Akash Raj, Keunwoo Choi, FrenchKrab, Moreno La Quatra, Piotr Želasko, Amiasato, Emmanuel Schmidbauer, Lasse Hanson, and Riccardo Miccini. asterood-teemm/oorhh-aaan http://tooobbaadionnntttttnnsss v0.11.1, 2024. URL https:
aedoo..or//00/ 1t..1..221//eeodoe 10628988.
Stefan Kahl, Connor M. Wood, Maximilian Eibl, and Holger Klinck. Birdnet: A deep learning solution for avian diversity monitoring. Ecological Informatics, 2021.
Toannis Kakogoorgion, Spyros Gidaris, Bill Psomas, Yannis Avrithis, Andrei Bursuc, Konstantinos Karantzalos, and Nikos Komodakis. What to hide from your students: Attention-guided masked image modeling. In European Conference on Computer Vision (ECCV), 2022.
16

---

Published in Transactions on Machine Learning Research (08/2025)
Juho Lee, Yoonho Lee, Jungtack Kim, Adam Kostorek, Seungjin Choi, and Yee Whye Teh. Set transformer: A framework for attention-based permutatioo--nvariaat neural networks. In International Conference Machine Learning (ICML), 2019.
Johannes Lehner, Benedikt Alkin, Andreas Fürst, Elisabeth Rumetshofer, Lukas Miklautz, and Sepp Hochreiter. Contrastive tuning: A little help to make masked autoencoders forget. Proceedings of the AAAT Conference on Artificial Intelligence (AAAT), 2024.
Markus Marks, Manuel Knott, Neehar Kondapaneni, Elijah Cole, Thijs Defraeye, Fernando Perez-Cruz, and Pietro Perona. A closer look at benchmarking self-supervised pre-training with image classification. International Journal of Computer Vision (IJCV), 2025.
Tyass Moummad, Romain Scrizel, Emmanouil Benctos, and Nicolas Farrugia. Domain-invariant reprosentation learning of bird sounds. XrX:::2409. 009.8899, 2024.
Amanda K. Navine, Richard J. Camp, Matthew J. Weldy, Tom Denton, and Patrick J. Hart. Counting the chorus: A bioacoustic indicator of population density. Ecological Indicators, 169, 2024. doi: 10.1016/j. 20tind.2024.11222
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Po-Yao Huang, Shang- Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learving Research (TMLR), 2024.
Kamalesh Palanisany, Yu-Wei Chao, Xinya Du, and Yu Xiang. Proto-clip: Vision-language protolypical network for few-shot learning. In IEEE/RRI RRSJ International Conference on Intelligent: Robots and Systems (IROS), 2024.
Danicl S. Park, William Chan, Yu Zhang, Chung-Chens Chiu, Barret Zoph, Ekin D. Cubuk, and Quoe V. Le. Specaugment: A simple data augmentation method for automatic speech recognition. In Interspeech (ISCA), 2019.
Namuk Park, Wonjae Kim, Byeongho Heo, Taekyung Kim, and Sangdoo Yun. What do self-supervised vision transformers Iearn? In International Conference on Learning Representations (ICLR), 2023.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. Xrv: 1at:::991.01001.1 01703, 2019.
Karol J. Piczak. Esc: Datasct for environmental sound classification. In ACM International Conference on Mullimedia (MM), 2015.
Lukas Rauch, Denis Huseljic, Moritz Wirth, Jens Decke, Bernhard Sick, and Christoph Scholz. Towards deep active learning in avian bioncoustics. arXiv:2406. 18621, 2024a. URL.
2406.18621.
Lukas Rauch, Raphael Schwinger, Moritz Wirth, René Heinrich, Denis Huseljic, Marek Herde, Jonas Lange, Stefan Kahl, Bernhard Sick, Sven Tomforde, and Christoph Scholz. Birdset: A large-scale dataset for audio classification in avian bioacousties. Xrx:22200 10380, 2024b. URL https://dol. wtt.////....000/22......... 2403.10380.
Tal Ridnik, Emanuel Ben-Baruch, Nadav Zamir, Asaf Noy, Itamar Friodman, Matan Protter, and Lihi Zelnik-Manos Asymmetric loss for multi-label classification. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
17

---

Published in Transactions on Machine Learning Research (08/2025)
David Robinson, Marius Miron, Masato Hagiwara, and Olivier Pietquin. Naturelm-audio: andio-language foundation model for bioacoustics. In International Conference on Learning Representations (ICLR), 2025.
Aaqib Saeed, David Grangier, and Neil Zeghidour. Contrastive Learning of General-Purpose Audio Repre- sentations. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3875-3879, 2021.
Julian C. Jhmmmer-Zmmmmmmmnnn, Vlad Demartsev, Baptiste Averly Kiran Dhanjal-Adams, Mathieu Duteil, Gabriella Gall, Marius FaiB, Lily Johnson-Ulrich, Dan Stowell, Marta B. Manser, Marie A. Roch, and Ariana Strandburg-Peslle
animal2vec and meerkat: A self-supervised transformer for rare-event raw audio imput and a large-scale reference dataset for bioacoustics. arXiv:2406.01253, 2024. URL https: //doi.org/10 10. 48550/ arXiv. 220..24466000 01253.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in Neural Information Processing Systems (NeurIPS), 2017.
Dan Stowell. Computational bioacoustics with deep learning: A review and roadmap. arXiv:2112.06725, 2021.
Mingxing Tan and Quoc V. Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning (ICML), 2019.
Hongduan Tian, Feng Liu, Zhanke Zhou, Tongliang Liu, Chengqi Zhang, and Bo Han. Mind the gap between prototypes and images in cross-domain finetuning. Advonces in Neural Information Processing Systems (NewrIPS), 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Maric-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. arxi::3302.139 2023.
Michael Tschannen, Manoj Kumar, Andreas Steiner, Xiaohua Zhai, Neil Houlsby, and Lucas Beyer. Image captioners are scalable vision learners too. Advances in Neural Information Processing Systems (NeurIPS), 2023.
Joseph Turian, Jordie Shier, Humair Raj Khan, Bhiksha Raj, Björn W. Schuller, Christian J. Steinmetz, Colin Malloy, George Tzanetakis, Gissel Velarde, Kirk McNally, Max Henry, Nicolas Pinto, Camille Noufi, Christian Clough, Dorien Herremans, Eduardo Fonseca, Jesse Engel, Justin Salamon, Philippe Esling, Pranay Manocha, Shinji Watanabe, Zeyu Jin, and Yonatan Bisk. HEAR: Holistic Evaluation of Audio Representations. In NewrIPS 2021 Competitions and Demonstrations Track, 2022.
Bart van Merričnboer, Jenny Hamer, Vincent Dumoulin, Eleni Triantafillon, and Tom Denton. Birds, bats and beyond: Evaluating generalization in bioacoustics models. Frontiers in Bird Science, 2024.
Willem-Pier Vellinga and Robert Planqué. The xeno-canto collection and its relation to sound recognition and classification. Tn CEUR Workshop Proceedings, 2015.
Matthew Walmer, Saksham Suri, Kamal Gupta, and Abhinav Shrivastava. Teaching maiters: Investigating the role of supervision in vision transformers. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling, In IEEE/CVF Conference on Computer Vision and Pollern Recognilion (CVPR), 2022.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. wrXYt:2205001917, 2022.
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations (ICI.R), 2018.
18

---

Published in Transactions on Machine Learning Research (08/2025)
Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv:21 11.07832, 2021.
19

---

Published in Transactions on Machine Learning Research (08/2025)
A Data Curation
This appendix details our data curation pipeline, component for both the training of our models and the evaluation of their performance. We apply distinct curation strategies for: (1) the large-scale pretraining dataset, (2) the few-shot learning subsets, and (3) the full datasets for downstream task evaluation. The specifics of each methodology are outlined below.
A.1 Pretraining Data
We derive our pretraining set from BirdSet's XCL collection using the provided file-level species labels, the event detector and the sampling algorithm from BirdSet (Rauch et al., 2024b). Since each recording may contain multiple bird-call events, we first split every recording into its detected events. To mitigate class imbalance, where some species have many events and others few, we enforce three constraints: (a) a maximum number of events per species, (b) per-recording event cap, and (c) at least. one event per recording. We implement these rules via simple sampling algorithm that iteratively trims over-represented recordings until all class and event limits per-recording event cap to 2. This leads to our XCL-1. 7M pretraining dataset. The uncurated dataset XCL-3. 4Mg contains all the detected events.
met. We set the number of maximum events per species to 500 and
A.2 Few-shot Data
For our few-shot learning evaluation, we construct k-shot subsets with h (1,5, 10) from each BirdSet downstream training split so that every species contributes exactly k audio clips (on file-level). To assess sampling variability, we generate three independent subsets per k using different random seeds. Because BirdSet's labels are weakly labeled, we prioritize recordings under 5 seconds to reduce label noise (i.e., increasing the chance that the annotated species on file-level is actually present each extracted event). In the following, we describe the k-shot subset creation pipeline:
1. Initial filtering: We first go through all recordings in the original training split for given BirdSet task. The preferred recordings are up to 5 seconds, aligning with the 5-second input file length of the model to mitigate label noise. However, since there are not always even 5-second recordings for each species, we also include 20 seconds samples, but only if they contained just one primary bird species (no secondary species listed).
2. Sample extraction from recording: From each selected recording (which contain multiple vocalization events), we extract individual 5-sccond audio samples centered around these events based on the given events from the BirdSet metadata. To avoid over-representing any single long recording, a recording yields multiple 5-second samples, one is randomly chosen as a primary sample for that recording, and the others are considered leftover samples.
3. Selecting k samples per class:
We first try to pick k samples from the primary sample associated with that species (if available). If there are not enough primary samples (less than k), we then try to fill the remaining spots using the leftover samples from that species. If, a class still has fewer than k samples, we do not fill up further from recordings that failed the initial filtering. This means some classes in the few-shot set might have fewer than k samples not enough suitable recordings are available. If a species has more than ie suitable samples, we randomly selected & samples from the available pool for that species.
4. Dataset construction: The selected & samples per class form the new train split for that specific k-shot, sooded dataset. The original BirdSet test split (5-sccond segments) is kept as is for evaluation.
A.3 Full Downstream Data
The BirdSet dataset provides filo-level (weak) labels for recordings that may contain multiple vocalization events, necessitating downstream processing to generate task-specific datasets (Rauch et al., 2024b). Our
20

---

Published in Transactions on Machine Learning Research (08/2025)
preliminary experiments small focal validation sets on all downstream tasks and models indicated that uniform sampling strategy across all BirdSet's tasks is suboptimal. Consequently, we adopted tailored approaches: For the datasets HSN, UHH, and NBP that have the lowest class counts across tasks, we utilized BirdSet's s inherent sampling strategy by adding a species cap of 500 and extracting a maximum of events per recording, following Rauch al. (2024b). More details are available in the BirdSet implementations. Conversely, the datasets SNE, POW, NES, PER, and SSW are characterized by larger data volumes and higher class counts. We found it more advantageous to employ our few-shot sampling approach with k 64 to further reduce label noise and class imbalance. This proved beneficial for experiments involving both frozen representations across all models. The final number of samples curated for each dataset in our study is detailed in the main text in Table 2.
and
B Metrics
This appendix provides a detailed description of the evaluation metrics employed to assess model performance throughout this study. We evaluate model performance in the main paper with the mean average precision (MAP) as the metric in multi-label classification (Huang et al., 2022; Chen et al., 2023; 2024a; Rauch et al., 2024b). In the additional results in Appendix E, we also report the area under the receiver operating characteristic curve (AUROC), and lop-1 accuracy (T1-Acc), following the multi-label benchmark from BirdSet.
MAP, also referred to as class-wise MAP (cmAP) (Rauch et al., 2024b), first calculates the average precision (AP) for each class independently and then computes the macro-average of these AP scores across all C' classes:
1 E AP(c).
MAP
MAP reflects the model's ability to rank positive instances higher than negative ones for each class across all decision thresholds, providing comprehensive assessment of retrieval performance. By averaging class-wise AP scores, MAP gives equal weight to each class, regardless of its prevalence in the dataset. While robust, it can be sensitive to classes with very few positive instances (van Merriënboer et al., 2024).
AUROC quantifies the model's ability to discriminate between positive and negative instances across all possible classification thresholds. It is equivalent to the probability that a randomly chosen positive instance ranked higher by the model than a randomly chosen negative instance (Heinrich et al., 2025). For multi-label setting, it often computed as the average AUROC across all classes:
1 & AUROC= = C Σ
1
ΣNX
Σ
16 Bm.ec) >mm.e)
NEY, mEY
where Y+,e and Y-, are the sets of indices for positive and nogative instances for class e respectively, In.e is the predicted score for instance n and class e, and I[(-) is the indicator function. AUROC is wrreshold-independdee and provides a balanced view of performance, where a random classifier yields an AUROC of 0.5 (van Merriënboer et al., 2024).
T1-Ace assesses whether the class assigned the highest predicted confidence score by the model is among the set of true labels for given instance (Rauch et al., 2024b):
T1-Acc
Σ [o((op) E Yn.). N
1
where N is the total number of instances, g/(op) is the class with the highest predicted score for instance Ya,: is the set of true labels for instance n, and T(.} is the indicator function. While not a canonical multi-label metric, T1-Ace offers an intuitive measure of whether the model's most confident prediction is correct, which is relevant in practical applications where identifying at least one present species is a primary goal.
21

---

Published in Transactions on Machine Learning Research (08/2025)
C Implementation and Infrastructure
To facilitate reproduction, experiments were under the following conditions: Models were trained and evaluated on compute cluster using NVIDIA L40s and A100 GPUs. CPU types included Intel Xeon Gold 6252 and AMD EPYC 7662, with nodes having approximately 600 GB RAM. The software environment consisted of Ubuntu OS, Python 3.9, PyTTrch (Paszke et al., 2019), and PyTorch Lightning (Falcon & The PyTorch Lightning team, 2019). Small-scale testing was performed on a workstation using an NVIDIA RTX 4090 GPU and an AMD Ryzen 7950X CPU.
D Model Training
This appendix outlines the specific training configurations and hyperparameters employed for our experiments, including both the ablation studies and the final benchmark evaluations.
D.1 Augmentations
Our data augmentation pipeline, appliod during to all experiments, including few-shot multi-label classification and probing variants, is adapted from the strategies outlined BirdSet (Rauch al., 2024b). We empirically tuned the application probability for each selected augmentation based on preliminary experiments on the validation data HSN. The spectrogram time- freeureeyy masking similar to SpecAugment (Park al., 2019). However, omit the time-warping component of SpecAugment, which did not improve validation performance in our setting. A key component of our pipeline is waveform-level mixup, implemented using Torctauudiomnntations (Jordal et al., 2024), which demonstrated superior performance compared to spectrogram-based or standard linear mixup. For augmentations requiring external audio, such as background noise addition and no-call mixing, we utilized environmental recordings sourced from BirdSet's vOx dataset. Additional waveform and spectrogram-level augmentations, along with their specific parameters, are detailed in Table
Augmentation
Probability
Parameters
Waveform-level augmentations
cyclic rolling start
1.0
multi-label mixup background noise
0.9
min-smr=2.0, max-snr=30.0, mix-target=mmion, max-samples=3
0.5
min-snr-3.. max-smr=30.0
colored noise
0.2
min-snrr3.0, max-snr-30.0
min-f-decay=22, max-I-decay=2
gain adjustment tment
0.2
muin-gain--Be 18, max-gain-6
no-call mixing
0.075
Spectrogram-leved augmentations
frequency masking
0.3
5000 iid-masks= True
time masking
0.3
pare-mask-param--tt tid-masks='Trus
Table 9: Data augmentation techniques and parameters applied during all experiments in the paper. This includes fine-tuning and probing on the complete dataset as well as few-shot, probing across all techniques.
D.2 Hyperparameters
This section details the hyperparameters used for fine-tuning our models in both the ablation studics and the main BirdSet benchmark experiments. These parameters, including learning rates, batch sizes, and optimizers, were empirically validated on the HSW dataset. After validation, the hyperparameters were fixed across all downstream tasks. All models utilize the asymmetric loss for multi-label classification (Ridnik et al., 2021). The same core settings were also applied the few-shot learning benchmark, with minor adjustments primarily to the learning rate and number of training epochs to suit the reduced data regime.
22

---

Published in Transactions on Machine Learning Research (08/2025)
For each model, whether we fine tune it on the full data probe frozen representations, validate two hyperparameters:: the learning rate and the weight decay. With prototypical pooling probing we additionally explore the number of prototypes SJ$ (see Table 13) and the learning rate of the prototype vectors. Every model and setting combination is trained for 30 epochs in the full data regime and 50 epochs in the few shot regime, using random search over these discrete grids:
Learning rate:
1x 4x0
1xx10-0)
Weight decay: {1x10 10 33x10 1,x 1,5x00 *}
Number of prototypes J: {5. 15, 20, 20)
Prototype learning rate: (2x10-3, 2,0 10-2, 2,0 10
23

---

Published in Transactions on Machine Learning Research (08/2025)
BirdAVES (Hagiwara, 2023)
SimCLR
Audio-MAE (Huang et al., 2022)
Parameter
Bird-MAE
(Moummad et al., 2024)
Model Training (Fine- Tuning / Probing on full data)
Input Type
Waveform
Spectrogram
Spectrogram (fbank)
Spectrogram (fbank)
Batch size
64
128
128
128,128,64
Epochs
30
30
Gradient clip
0.5
0.1
2
2
Precision
16-mixed
16-mixed
1-mmixed
16-mixed
Learning rate
1e-5
4e-4
3e-4
3e-4
Optimizer
AdamW
AdamW
AdamW
AdamW Asymmetric
Loss
Asymmetric
Asymmetric
Asymmetric
Weight decay
le-4
3e-4
3e-4
30-4
Layer decay
0.75
0.75
Pooling
mean Cos- Annealiig
prototypical Cos-Annealing
protolypical
Scheduler
Cos-Annealing
Cos-Ann AeoeAnncaling
Model Training (Few-shot probing)
Learning Rate
4o-4
Epochs
50
Processing
fft
1024
~000
~00
Mels Sampling Rate
128
128
16 kllx
16 kllz
16 kllz
32 klix
Norm Mean Std
0.5347
-4.2
-7.2
0.0772
4.57
4.43
Fmin-Fmax
50-8000 Hz
full band Hanning ms log FBANK
full band Hanning
Window Type
Hanning
Frame Shift
samples AmplitudeToDB
10 ms log FBANK
Log Scale
Pretrodzing
Type
SSL HuBERT-L
SSL
SSL ViT-B/16
SSL VIT- (B.-(B,,,H)II /16 86,307,632
Architecture
CvT-13
Parama [M]
317
20
Dataset
Xeno-Canto
Xeno-Canto"
AS-2M
XCL-1.7M
Prototypical Pooking Probing
Prototypes
20
20
20
20
Protoype LR
4e-2
4e-2
4e-2
4e-2
Focal Similarity
True True
True
True
True
Orthogonality Loss
True
True
True
Attentive Pooling
Probing
Attention heads
12
12
Table 10: Training hyperparameters for models evaluated in this paper. These settings cover evaluations using frozen representations, full fine-tuning (ablation studies), and both techniques for multi- label classification on the benchmark results. For the multi-label few-shot classification benchmark, we largely retained the same hyperparameter settings, with specific adjustments primarily to the learning rate and number of training epochs.
24

---

Published in Transactions on Machine Learning Research (08/2025)
E
Additional Benchmark Results
This appendix presents complementary results to the main multi-label and few-shot classification benchmarks discussed in the main text. These additional evaluations provide further insights into model performance with BirdSet's metric suite.
E.1
Multi-Label Classification
Fine-Tuming
Prototypical Probing
Linear Probing
Model
AUROC
MAP
TI-Ace
AUROC
MAP
T1-Acc
AUROC
MAP
T1-Aoc
BirAAES
0.86 + 0.03
0.43 : 0.01
0.35 + 8:03
0.79 + 0.00
0.33 4: 0.00
0.25 + 0.01
0.81 :: 0.00
0.15 = 0.00
0.13 # 0.00
SinCLR
0.53
0.39 0.00
0.31
0.73
0.18
0.70 0.00
0.17
0.12
Audio-MAE
0.53 +: 0.00
0.19 4: 0.00
0.10 4 0.00
0.19 4 0.00
0.10 4: 0.00
0.10 +: 0.00
0.68 4: 0.00
0.09 0.00
0.06 +: 0.00
Bard: B
0.50
0.52 0.01
0.01
0.89
0.41
0.16
0.77
0.00
Bird-MAE
0.90 + 0.00
0.55 4: 0.00
0.64 4: 0.01
0,90 +: 0.00
0.49 1: 0.01
0.38 +: 0.01
0.72 4: 0.00
0.12 0.00
0.07 +: 0.00
Bird-MAE-1
0.91 0.00
0.55
0.00
0.65 + 0.00
0.00
0.48 0.01
0.46 0.08
0.76 0.00
0.00
0.10 +
DardAVES
0.60 + 0.01
0.11 + 0.02
0.10 + 0.00
0.66 + 0.00
0.20 + 0.01
0.28 + 0.03
0.68 + 0.01
0.15 0.00
0.31 + 0.04
StoCLR
0.81 + 0.30
0.35 0.01
0.67 + 0.04
0.64 0.01
0.17 0.00
0:27 + 0.05
0.71 0.01
0.18
0.00
0.39 + 0.01
Audio-MAE
0.82 = 0.01
0.39 0.01
0.75 + 0.00
0.74 = 0.00
0.20 + 0.00
0.26 + 0.02
0.60 0.00
0.10 0.00
0.09 : 0.01
B-B
0.86 + 0.30
0.45 0.01
0:83 + 0.00
0.85 0.01
0.38 0.01
0.74 + 0.02
0.67 0.00
0.13 0.00
0:23 + 0.06
Bird- M
0.90 + 0.00
0.55 0.01
0.91 +: 0.00
0,90 + 0.00
0.52 0.00
0.89 + 0.01
0.67 0.01
0.11 0,00
0.22 + 0.06
MAE- H
0.89 4 0.00
0.54 0.01
0.89 1 0.00
0.89 1: 0.01
0.50 1: 0.01
0.56 1: 0.01
0.67 1: 0.00
0.15 0.00
0.32 1: 0.04
BadAAES
0.75 1: 0.03
0.06 0.00
0.13 1 0.02
0.65 1 0.01
0.12 0.01
0.23 1: 0.03
0.70 0.00
0.08 0.00
0.12 1: 0.02
SinCLR
0.85 +: 0.01
0.24 0.01
0.39 +: 0:01
0.62 :: 0.01
0.08 4: 0.01
0,20 +: 0.02
0.75 +: 0.00
0.11 = 0.00
0.11 + 0.01
0.87
0.30 0.00
+ D:01
0.87
0:00
+
0.70 0.00
0,00
0.07
BAE-
0.90 = 0.00
0.37 = 0.00
0.47 1 0.00
0.92 = 0.00
0.25 1: 0.00
0.41 + 0.01
0,78 2: 0.00
0.08 = 0.00
0.11 = 0.01
Bid-MAE-L
0.91
0.00
0.41 0.01
0.52 + 0.00
0.93 0.00
0.38
0.47 0.00
0.75 0.00
0.06 0.00
0.22 0.01
Bird-MAE-H
0.89 + 0.00
0.39 : 0.00
0.51 + 0:01
0.92 - 0.00
0.36 +: 0.01
0.47 +: 0.01
0.75 : 0.00
0.11 0.00
0.22 : 0.01
BidAVES
0.65
0.07 0.00
+ 0:04
0.59
0:01
+
0.67 0.01
0.00
0,20 0.01
SinCLR
0.75 + 0.01
0.16 : 0.00
0.40 1 0:03
0.58 = 0.00
0.09 1: 0.01
0.12 + 0.04
0.61 = 0.01
0.09
0.00
0.18 = 0.03
SME
0.81
0.22 0.00
0.01
0.75
0.00
0.62 0.01
0.00
0.01
BiE--MAE B
0.53 2 0.01
0.28 2 0.01
0.32 1 0.01
0.81 2 0.00
0.22 1 0.00
0.35 1 0.12
0.67 1 0.01
0,118 = 0.00
0.01 I 0.01
Bird- L
0.00
0.34 0.00
0.61 1: 0.01
0.00
0.30
0.54 0.01
0.64 0.01
0.08 0.00
0.20 0.01
Bir--MAE-1
0.85 + 0.01
0.32 0.00
0.62 + 0.01
0.85 + 0.01
0.28 + 0.01
0.48 + 0.02
0.61 0.01
0.09 0.00
0.18 + 0.03
DirdAVES
0.76 0.02
0.04 0.00
0.10 + 0.02
0.62 0.00
0.08 0.00
0.20 + 0.03
0.73 0.00
0.06 0.00
0.16 +
SinCLR
0.83 + 0.02
0.19 0.02
0.44 + D:01
0.00 + 0.01
0.05 0.00
0.16 + 0.02
0.74 0:00
0.07 0.00
0.18 + 0.01
SSW
Audio-MAR MAE
Bird-MAF-B
0.89 = 0.00
0.88 + 0.00
0.27 0.00
0.33 0.01
0.52 + 0.01
0:62 + DO0
0.94 + 0.00
0.56 = 0.00
0.09 1: 0.00
0:23 + 0.00
0.25 + 0.02
0.49 + 0.00
0.77 0.00
0.69 0.01
0.06 0,00
0,08 = 0.00
0.10 : 0.01
0.14 + 0.00
Bird-MAE-L
0.93 = 0.00
0.41 0.00
0.70 1 0.00
0.94 : 0.00
0.38 0.00
0.82 + 0.00
0.77 0.00
0.06 0.00
0.14 # 0.00
Bird-MAE-H
0.91 0.00
0.41 0.00
0:68 + DO0
0.94 + 0,00
0.36 0:02
0.60 + 0.02
0.74 0.00
0.00 0.07
0.18 + 0.01
BadAVES
0.58 1 0.02
0.05 0.01
0.11 1 0.02
0.54 4 0.00
0.05 0.00
0.14 1: 0.00
0.60 1: 0.00
0.06 0.00
0.13 1 0.01
SimCLR
0.73 0.00
0.17 0.01
0.43 1: 0.01
0.52 0.00
0.03 0.00
0.06 +: 0.02
0.63 0.01
0.06 0.00
0.13 4 0.01
Mudi
0.77 1 0.00
0.21 1: 0.00
0.49 1 0.0n
0.71 4 0.00
0.09 1: 0.00
0.17 1 0.00
0.60 0.00
0.N4 = 0.00
0.09 1 0.00
PER
Bard 15
0.17 =: 0.00
0.28 0.00
0.34 1 0.00
0.79 0.00
0.21 1211:00 0.00
0.49 1 0.01
0.63 0.00
0,05 0.00
0.13 1 0.00
B-
0.82 + 0.00
0.35
0.00
0.60 + 0.01
0.82 + 0.00
0.31 0.00
0.59 + 0.00
0.63 + 0:01
0.06 0.00
0.13 + 0.00
Bird-MEE-1
0.81 + 0.00
0.33 0.00
0.59 4: 0:00
0.82 0.00
0.30 0.01
0.58 + 0.01
0.66 0.01
0.07
0.00
0.18 à: 0.02
BidAAES
0.82 +: 0.01
0.22 : 0.01
0.47 + 0.02
0.71 + 0.01
0.15 :: 0:01
0.29 + 0.00
0.72 + 0.00
0.12 = 0.00
0.25 + 0.00
SinCLR
0.76
0.21 0.00
1
0.57
0.03
0.00
0.15
0,29 0.00
Audio-MAN
0.82 + 0.00
0.26 +: 0.00
0.38 + 8:00
0.74 + 0.00
0.17 +: 0.00
0.26 + 0.00
0.67 + 0.00
0.11 0.00
0.29 + 0.00
UEN
Bird: M-E-B
0.81
0.28 0.01
1
0.01
0.03
0.00
0.15
0,29 0.00
Bird-MAE
0.82 à: 0.01
0.30 d: 0.00
0.42 1: 0.00
0.83 à 0.00
0.30 1: 0.00
0.36 4: 0.00
0.76 1: 0.00
0.15 0.00
0.28 4: 0.00
B
0.50
0.30 0.00
0.82
0.00
1:
0.71 0.05
0.08
DirdAVES
0.93 +: 0.00
0.70 # 0.00
0.78 + 0.01
0.79 + 0.02
0.40 +: 0.01
0.45 +: 0.01
0.79 + 0.00
0.34 0.00
0.37 # 0.01
StnCLR
0.82 0.00
0.60 0.00
0.73 + 0:01
0.73 0.01
0.27 0:01
0.30 + 0.00
0.73 0.00
0.34 0.00
0.37 + 0.00
Audio-MAE
0.93 + 0.00
0.67 0.01
0.68 + 0:01
0,81 + 0.00
0.35 + 0.00
0.37 + 0.00
0.75 0.00
0.25 0.00
0.21 + 0.00
0.91
0.50
0.63 0.01
0.66 + 0.02
0.92 0.03
0.63 0:07
0.65 + 0.06
0.79 0.00
0.34 0.00
0.45 + 0.01
MArd-MAE-L
0.94 = 0.00
0.72
0.00
0.72
1 0.01
0.92 0.00
0.69 1: 0.00
0.69 : 0.01
0.92 0.00
0.42 0.00
0.12 : 0.00
Bird-MAE-H
0.94 + 0.00
0.69 0.00
0.70 + 0:01
0.92 0.01
0.60 0.02
0.69 + 0.02
0.80 0.00
0.25 0.00
0.21 + 0.00
Table 11: Fine-tuning, prototypical probing and linear probing results on BirdSet's malti-label classification benchmark (MAP, AUROC, T1-Acc.). Comparison of SSL models with full training data, following the evaluation protocol of BirdSet. Best results are highlighted. This complements Table 6 from the main text.
25

---

Published in Transactions on Machine Learning Research (08/2025)
E.2
Few-Shot Multi-Label Classification
I-chint
G-shot
10-ahot
AUROC
MAP
AUROC
MAP
Tl-Acc
AUROC
Tl-Axc
Proto
72.05 0.86
25.22 1.56
45.15 # 7.38
88.76 = 0.36
41.46 =: 0.53
74.37
1.11
86.25
0.36
42,96 # 0.60
87.61 + 0.53
POW
Linear
62.26
1.73
7,06
4: 0.36
9.90 9.48
67.01 4: 0.26
11.25 0.32
7.26
3.80
62.05 0.49
12.53 + 0.28
26.15 10.65
Attenitive
1.41 57.45
10.28
0.83
6.14 4.26
70.35
1.20
23.11
0.70
8.62 54.30
75.80
31 0.40
2.01 76.88
Proto
60.55 7.23
9.91 + 4.44
2,03 1.34
82.64 1.85
27.88
3,91
16.95
6.26
86.35
0.56
38.66 + 2.08
21.58 2.02
Linear
52.16
0.25
1: 1.17
3.57
1: 0.23
1.15
Attendtive
2.82 60.03
6.48 + 0.90
3.20
3.22
71,26 + 1.75
19:01
2.75
11.07 3:01
80:25 + 0.94
31 18
+ 2.36
22.01 + 2.67
Proto
70.50
0.69
14.36
0.50
18.81 4. 7.67
80.00 0.37
26.83
0.40
58.94
2.42
81.22
0.14
29.33 1: 0.11
62.10 1.26
Linear
12.96
1.31
2.49
0.05
4.80
5.36
60.00 + 1.00
4.00
4.94 2.14
5.41 0.12
7,02
0.70
Atteutive
$5.89
1.03
3.17 +: 0.24
1.12 2.15
68.45 +: 1.53
14.06 à: 0.56
39.55 à: 2.15
74:22 à: 1.06
20.09 +: 1.08
30.83 A: 2.29
Proto
1.27
12.03
0.93
13.70
1.56
88.97
28.65
1,02
2.10
91.44
0.31
33.86
46,91
0.42
MES
Linear
55.96 + 1.73
1.06 + 0.17
0.29
0.17
64.77 + 0.97
3.65 + 0.31
2.64 + 1.17
68.96 + 0.73
0.71
0.11
7.82 = 0.54
Adtenttvve
62.20 1.52
3.58 +: 0.46
1.57 2.80
63.36 +: 0.90
20.96 0.13
32.76
4.37
87:76 0.76
28.78 +: 0.64
42.53 1.44
Proto
67.89
1.30
9.82
10.59 22.02
73.73
14.96
0.52
35.36
0.44
75.24
0.63
21.06
48,78
UHI
Linear
57.27
2:36
5.54 1. 0.09
8.02 1: 12:29
58.21 1. 1.85
6.04 1. 0.36
22.25
1. 5.30
60.18 1. 2.22
5.74 0.36
17.72 1. 7.57
Atteutive
60,62
0.86
7,20 0.85
8.74 2,80
65.01 1.01
9.26
0.30
17.74 5.48
06.57
13:00
1.53
45.12 2.22
Proto
69.10
0.94
22.81
1.35
17.89 1: 3.92
84.51 :: 0.71
47.81
0.60
49.13
2.81
86.37 1.11
54.58 :: 1.60
55.88 1: 1.26
MRP
Linesr
13.09
0.35
1.44 + 0.26
4.08 0.40
58.00 + 0.51
8.18 + 1,02
4.33 + 0.76
61.06 + 0.24
11.00 + 1.06
7.73 0.80
Atteutive
58.05
1.02
9.08 0.87
0.95
17.83 + 0.74
36.80
1.63
41.25
2.22
82.58
47.06 + 0.53
1.79 55.33
Proto
61.33 1: 2.53
12.61 1: 0.68
26.87
1: 3.96
90.08 :: 0.28
25.87 1: 0.80
45.11 1: 0.98
92.54 2: 0.17
32.64 :: 0.13
50.01 1: 0.54
SSW
Linear
01.94
2.05
1.15 1 0.09
0.38 0.32
07,25 4: 1.05
2,35 0.23
0.07 4 3.48
68.00
3.68 0.30
12:16 0.29
Assentive
55.86
0.95
2:25 1: 0.64
3.12
1.53
79.93 1: 0.87
18.16 1. 0.73
38.35 0.54
83.81 1 0.48
27 1: 0.65
49.30 1: 1:27
Proto
68.86 + 0.90
9.05 #: 1.66
7.23 + 3.56
81.03 = 0.25
22.63
0.20
38.15
0.40
83.01
0.74
25.60 + 0.19
47.16 # 2.48
Linoar
54.18
0.27
66.28 1.06
4.63
2.38 1,27
58.56
4.40
Attontiiv
53.39 2.00
4.56 +: 0.33
2.80 1.61
68.06 + 1.80
13.01 + 0.49
27.63 +: 2.57
75:90 + 0.97
10:43 + 1.31
37.30 + 4.32
Table 12: Frozen representation results for prototypical, linear and attentive probing on our few-shol mullt--laeel classificution benchmark (MAP, AUROC, T1-Acc.). Comparison of our best-performing Bird-MAE-L model with few-shot training data, following the evaluation protocol of (Rauch et al., 2024b). Best results are highlighted. This complements Table 7 from the main text.
F Additional Ablations
This appendix contains supplementary ablation studies. As in the main text, all ablations are performed on the HSN validation set with the best-performing model modifications and parameters.
J
Probing
Fine-tuning
5
39.16 16
52.22
10
43.78
54.07
15
47.47
54.16
20
49.92
54.91
25
49.92
54.91
30
49.97
54.71
Table 13: Ablation on number of prototypes (J) on HSN with the Bird-MAF-L, model (MAP%).
G Generalizability Study on MeerKAT
To investigate the generalizability of our findings beyond avian bioacoustics, we conduct preliminary study on the MeerKAT datasct (Schäfer-iimmerman et al., 2024), which contains multi-label meerkat vocalizations. This small-scale study was designed to test two hypotheses: (a) that our domain-specific Bird-MAE provides transferable benefits to other fine-grained bioacoustic tasks, even when used as a frozen feature extractor, and (b) that prototypical probing remains superior to linear probing in this new domain.
Dataset and preprocessing. The MeerKAT dataset consists of 10-second multi-label andio clips with cleven unique call types originally sampled at 8kHz. As the dataset does not provide an official train-test. split,
26

---

Published in Transactions on Machine Learning Research (08/2025)
follow the protocol from Schäfer-Zimmernnnn et al. (2024) by randomly sampling 20% of the data to create a test dataset. To ensure efficient experimentation while demonstrating relative performance gains, we randomly sample 50% of the remaining data for our training set with a small validation split. During data loading, all audio recordings were upsampled to to match the pretraining specifications of our models. For compatibility with Bird-MAF's 5-second input window, the 10-second clips from the MeerKAT dataset were segmented into two non-overlapping 5-second chunks.
Training and augmentation. For this study, evaluate frozen representations from both the general- purpose Audio-MAE and our domain-specifi Bird-MAE. We apply linear probing and our proposed proto- typical probing to both models. The hyperparameters for the probing heads (e.g., learning rate, number of prototypes) are kept consistent with those used for the main experiments. We use minimal augmentation strategy, applying only multi-labe mixup with a probability of 0.5 during training.
27


Reviews:
[{"review_id": "vMaELgvnAk", "reviewer": "Reviewer_XGfz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper tackles the domain gap in applying general-purpose self-supervised audio models, such as Audio-MAE, to fine-grained tasks like bird sound classification. The authors propose Bird-MAE, a domain-specific masked autoencoder pretrained on a curated subset of BirdSet, and introduce prototypical probing as an efficient method to use frozen MAE representations. Across eight BirdSet benchmarks, the proposed approach achieves strong gains over both supervised baselines and previous self-supervised models. The improvement from frozen features using prototypical probing is particularly noteworthy, outperforming linear probing by a large margin and narrowing the gap to full fine-tuning to just a few points in MAP. The authors also provide thorough ablations on pretraining data, masking, pooling methods, and probing strategies, which gives the paper a strong empirical backbone. The release of code, pretrained models, and the new few-shot benchmark further enhances the paper’s value to the community.\n\nWhere the paper is slightly weaker is in terms of technical novelty. The backbone architecture (ViT-MAE) is taken directly from prior work, and while the prototypical probing idea is well-executed, it draws heavily from existing ideas in prototypical networks and attentive pooling. The probing layer is essentially a non-parametric readout module with class-wise centroids, and while the efficiency and effectiveness are appreciated, the novelty is modest. Additionally, the evaluation is focused solely on bird sound data — which makes sense given the paper’s goal — but leaves questions about how transferable the insights or methods are to other domains, such as speech, environmental sounds, or multi-modal settings.\n\nOverall, this is a strong applied paper that clearly demonstrates how to adapt self-supervised learning pipelines to challenging, fine-grained audio domains. While it doesn’t introduce new core learning principles, it’s well-motivated, carefully designed, and thoroughly evaluated, with takeaways that are likely to be useful for others working in low-resource or bioacoustic settings.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents Bird-MAE, a domain-adapted masked autoencoder framework for bird sound classification. Recognizing the limitations of general-purpose audio MAEs (e.g., Audio-MAE trained on AudioSet), the authors propose a holistic adaptation across three modules: (M1) pretraining on curated BirdSet data with optimized SSL recipe, (M2) fine-tuning with domain-specific augmentations and a novel prototypical pooling, and (M3) a parameter-efficient prototypical probing method that enhances the use of frozen features. The model achieves state-of-the-art (SOTA) performance on the BirdSet multi-label benchmark and demonstrates strong few-shot learning capability. Notably, prototypical probing narrows the performance gap to full fine-tuning to ~3%p, outperforming linear probing by up to 37%p in MAP.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify what’s truly novel in prototypical probing. The proposed probing method is effective, but it builds directly on prior work (e.g., prototypical networks, class token readouts, and attention-based pooling). It would help if the authors could more clearly articulate what makes their version different — is it the spatial max-pooling? the constrained linear projection? the choice of cosine similarity? Right now, it’s not obvious whether this is a new method or a clever reuse of existing pieces.\n\n2. Discuss generalization beyond birds. The paper focuses entirely on bird vocalizations, which is reasonable for a domain-specific model — but it would still be helpful to add some discussion (or preliminary experiments) on whether the benefits of prototypical probing or the Bird-MAE setup could carry over to other fine-grained audio tasks (e.g., dialect classification, insect calls, or low-resource speech). Even a negative result would be informative.\n\n3. Expand prototype interpretability analysis. Figure 5 is a nice start, but it’s quite limited. Since this paper touches on a domain (bioacoustics) where interpretability can actually matter — e.g., for ecologists or conservationists — it would be worth digging a little deeper. Are prototypes mapping onto specific call types? Harmonic structures? Species clusters? Even one or two examples would enrich the narrative.", "broader_impact_concerns": "I do not have major ethical concerns with this submission. The authors focus on improving bird sound classification through domain-adapted self-supervised learning, using publicly available datasets and releasing code/models to the community. These efforts can support positive use cases in ecology, biodiversity monitoring, and conservation, especially given the model's strong few-shot capabilities which could help in low-resource regions or with rare species.\n\nThat said, as with any audio recognition model trained at scale, it is worth noting that deployment in real-world monitoring systems (e.g., in forests, reserves, or urban spaces) raises general concerns about surveillance, unintended data capture, and responsible use. While not specific to this paper, future applications should involve clear usage guidelines and transparency for communities affected by automated sensing systems.\n\nOverall, the paper adheres to standard ethical practices, and the broader impact is likely to be positive, particularly in the context of open science and ecological research.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "K5RdXAppmg", "reviewer": "Reviewer_WHLn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n* Training code open, inference code open, model open.\n\n* Comprehensive experiments, very detailed ablations (I very much like Figure 3).\n\n* Good improvements specifically for the studied downstream task.\n\n\nWeaknesses:\n\n* The introduction argues against the high cost of fine-tuning general-purpose models but then proposes a solution  (building a domain-specific model from scratch) that is vastly more resource-intensive. This creates a logical inconsistency in the paper's motivation. The \"efficiency\" of their final solution lies at the inference and downstream adaptation stage (via cheap probing), but this comes at the cost of a massive up-front pretraining investment.\n\n* The paper shows that Audio-MAE fails on this fine-grained task but doesn't deeply discuss why. It jumps to the conclusion that domain-specific SSL is the necessary solution without exploring other hypotheses. Perhaps the Audio-MAE model is simply too small, and having a larger generic audio model would not have this problem. Or the MAE reconstruction task itself is less suited for learning sparse, fine-grained features like bird calls from a general dataset compared to other SSL methods (e.g., contrastive learning, or BYOL).\n\n* Novelty of prototypical probing: It seems that Heinrich et al. (2025) in AudioProtoPNet proposed a very similar method. This is claimed as one of the core novel contributions here, but it's not really novel.\n\n* Does not study how much the suggested changes (e.g. more epochs, mixup, etc) help on other downstream tasks from AudioSet.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "* Show that Audio-MAE, even though it is trained on a wide range of diverse audio, including bird sounds, is substantially worse for bird sound classification, compared to a model which is specifically tuned for bird sounds.\n\n* Proposes a model training pipeline specifically to train an audio MAE model which is good for classifying bird sounds. The most important addition is to apply bird-specific augmentations. All other changes also help and add up, but are minor compared to the bird-specific augmentation (even BirdSet instead of AudioSet only helps a little). This results in a new model: Bird-MAE.\n\n* Prototypical probing is compared to linear probing on top of Bird-MAE and other models for the downstream task benchmark.\n\n* A new type of benchmark on top of BirdSet for few-shot multi-label classification, where only very few (1, 5 or 10) training examples are used.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Reframe the motivation and efficiency argument in the introduction. This should explicitly acknowledge the high one-time cost of pretraining Bird-MAE. I would argue, having a general base model and only needing fine-tuning would be more efficient.\n\nDiscuss about alternatives, e.g. how to improve a general audio model to better handle such niche tasks.\n\nRephrase the contribution/novelty on prototypical probing: Stating that this is applied here, but not giving the impression that this is totally novel. Maybe only saying that it is novel to apply this for probing specifically?\n\nDescribing more directly/exactly what the difference of prototypical probing is compared to AudioProtoPNet in Heinrich et al. (2025).\n\nEvaluate also on some other (non-bird) test sets to see how much the methods (e.g. more epochs, mixup, etc) help there.\n \n\nSome more detailed thoughts:\n\nI would think, generic Audio-MAEs are maybe not big enough, so they only learn the most common present information. Or maybe the used SSL method is not good in catching features which are not so common in the data such as bird sounds, and other SSL methods could be better? This is maybe beyond the scope of this work to study this, but at least this should be discussed.\n\nIntro \"Adapting a general-purpose model to new domains via fine-tuning is resource-intensive, requiring high computational resources and extensive labeled data\": Yes, but training a new model from scratch is even more expensive? Or you mean it is better if the general-purpose model does not need fine-tuning but it's already good enough as-is? (That's what you mean by \"Leveraging frozen representations with lightweight probes\"?) Sure. But then you go on and argue \"...  it calls for a domain-aware adaptation of the entire SSL pipeline, including pretraining, fine-tuning, and frozen feature utilization.\" - so this is even much much more expensive. So I don't really understand the argument/motivation about efficiency, when the proposed method is doing exactly the opposite? If efficiency would be the goal, I would study how to make the general-purpose model (or SSL method) good enough / better to also handle such niche domains better. Not just to say that one specific general-purpose model can't do it well (then incorrectly generalizing from that that all general-purpose models can't do that well), and saying it needs domain-specific SSL.\n\n\n\nOther random comments:\n\n\nI don't know what this %_p notation means. I have never seen that. I would avoid it, as this just adds confusion. Or at least explain it somewhere, and give some reference where this comes from.\n\nMAP is a non-standard abbreviation. While reading the abstract, I was thinking about maximum a posteriori (MAP), but the sentence did not really make sense to me. It seems you mean \"mean average precision\" (MAP). This abbreviation should not be used in the abstract, or at least not without explaining it.\n\n\nIn speech recognition, SpecAugment is an extremely helpful augmentation. Was this tried? Would this help here?\n\n\n\n\nPS: I found it funny that in your Slurm train script, you write \"Error: srun failed. Sleeping for 2 hours...\", but then you sleep for 6h.\n\nPPS: I saw JEPA in your code. You don't write anything about that. I guess you tried it but it does not work yet? Maybe you can add some comments in the appendix about \"some preliminary JEPA experiments\"?", "broader_impact_concerns": "-", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "cxGT6F2KfZ", "reviewer": "Reviewer_RnKm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The strengths of the paper are:\n\n1. SOTA results compared to previous supervised and SSL methods.\n2. Systematic study of using SSL for bird classification, including pre-training, finetuning and representation probing.\n3. Novel prototypical probing and establishing few-shot benchmark for bird classification\n\nI find the paper is well written. I don't find obvious weaknesses for the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the effectiveness of self-supervised learning for bird classification tasks. The contributions of the work include: 1)\nadapting the MAE pipeline for bird sound classification; 2) proposing prototypical probing to enhance the utility of frozen MAE representations in bird sound classification; 3) establishing a novel few-shot multi-label classification benchmark. The paper achieves SOTA results on the BirdSet and the performance is much better than the supervised baseline.\n\nAlthough SSL has been previously studied in bird classification, the paper has thoroughly investigated the potential in SSL and largely improved the baselines based on the results.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I find the paper is well written. I don't see any major issues.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)