AADD v3.1: which audio degradation, and where

Code, CLI, Python API and interactive dataset report: github.com/gertelrina/aadd.

AADD (Automatic Audio Degradation Detection) finds 12 kinds of degradation in speech recordings every 20 ms: background noise, music, reverberation, clipping, low-pass / narrow band, high-pass, codec artefacts, packet loss, mains hum, clicks, pitch shift and time stretch. Several can be present at once. It is meant for cleaning speech datasets (ASR, TTS) and for auditing recordings.

• Backbone: Whisper-small encoder (88 M parameters), fine-tuned end to end
• Output: one sigmoid per class per 20 ms frame on 8-s windows; any length via sliding windows
• Languages seen in training: English and Russian

What changed in v3.1 (2026-11-06)

• What it is: v3.0 fine-tuned with about 36 h of real environmental noise from FSD50K (CC0 / CC BY clips only, no music or voice labels; clip list in eval/fsd50k_noise_pool.json.gz) and synthetic crowd babble added to the noise class: a 10 000-step run (lr 1e-5 encoder / 3e-4 head, 500 warm-up steps, cosine, seed 1) of which the step-4 000 checkpoint with the best validation AP is v3.1.
• Why: v3.0 called real background babble in cafés, streets and buses "music".
• Effect on VoiceBank-DEMAND, share of noisy clips with a "music" detection, by noise type:
  • café: 40 % → 13 %
  • public square: 46 % → 20 %
  • bus: 37 % → 17 %
  • office: 3 % → 2 %
  • living room: 85 % → 86 %
  • all 824 noisy clips: 42 % → 28 %; DNS-2020 noisy clips: 23 % → 16 % (clip score ≥ 0.5, eval/music_false_positives.json) — this noise contains sustained harmonic tones (music / TV), so the detection is probably correct.
• Side effect: noise is detected more often (café 84 % → 95 %, public square 55 % → 78 %).
• Synthetic held-out tests: unchanged or better (frame macro-AP 0.955 / 0.960, English / Russian).
• Which to use: Detector("v3") and --model v3 load v3.1; v3.0 stays available.

Usage

pip install "aadd @ git+https://github.com/gertelrina/aadd"

aadd scan my_dataset/ --report report.html              # downloads this checkpoint on first use
aadd scan call.wav --resolution 0.1 --segments per_100ms.csv        # v3.1 is the default model

from aadd import Detector

det = Detector("v3", resolution=0.1)       # segments of 0.1 s (1.0 by default); 20-ms frames in res.frame_probs
res = det.predict_file("call.wav")
res.detected                                # {'noise': 0.97, 'reverb': 0.88}
res.frame_probs.shape                       # (n_frames_20ms, 12)

Or download the file directly:

from huggingface_hub import hf_hub_download
path = hf_hub_download("gertelrina/aadd", "aadd_v3_1_whisper_small_frames.pt")
det = Detector(path)

Results

Public noise and reverberation benchmarks, not used for training (constructed mixtures of studio speech with real noise and real or simulated rooms; Dynamic-SUPERB draws on the same MUSAN / RIRS_NOISES corpora as training). Clip score = 95th percentile of frame probabilities.

set	class	ROC-AUC	recall at p ≥ 0.5	false alarms on clean files, same class at p ≥ 0.5	music detections at p ≥ 0.5 (no music present)
VoiceBank-DEMAND test, 824 noisy vs 824 clean twins	noise	0.992	0.60	0.0 %	28 % of noisy clips (v3.0: 42 %)
DNS Challenge 2020 synthetic dev-test, 600 clips	noise	0.998	0.91	0.0 %	16 % of noisy clips (v3.0: 23 %)
DNS Challenge 2020 synthetic dev-test, 600 clips	reverberation	1.000	1.00	0.0 %	—
Dynamic-SUPERB noise detection (VCTK + MUSAN)	noise	0.992	0.96	2.2 %	0.5 %
Dynamic-SUPERB reverberation, small / medium / large room	reverberation	0.994 / 1.000 / 0.996	0.98 / 1.00 / 0.97	0 %	0–1.5 %

A clip is flagged when the 95th percentile of its frame probabilities is ≥ 0.5 (the default threshold). Ranking is near perfect; at the default threshold the model is conservative on mild noise (VoiceBank-DEMAND SNR 2.5–17.5 dB), so lower the threshold when filtering data.

Held-out synthetic test sets (unseen speakers, noises and rooms; up to three overlapping degradations with random onsets):

test set	frame macro-AP (20 ms)	1-s segment macro-F1	clip macro-AUC
English (LibriSpeech test-clean), 3 000 × 8 s	0.955	0.905	0.989
Russian (Russian LibriSpeech test), 3 000 × 8 s	0.960	0.881	0.987
English, real measured RIRs	0.954	0.902	0.991
English, MIT IR survey RIRs	0.946	0.883	0.990

Frame AP per class (English / Russian): pitch shift 0.999 / 0.993, time stretch 0.999 / 0.994, low-pass 0.996 / 0.994, clipping 0.993 / 0.997, packet loss 0.991 / 0.992, high-pass 0.991 / 1.000, music 0.985 / 0.990, reverberation 0.985 / 0.991, clicks 0.976 / 0.979, codec 0.925 / 0.961, background noise 0.898 / 0.799, mains hum 0.726 / 0.825.

Speed (end to end: decoding, features, model; one process): 23 ms per second of audio on CPU (43× real time, about 84 s per hour of audio) and 6.5 ms on the Apple GPU (MPS), measured on an Apple MacBook Pro M3 Pro (6+6 CPU cores, 18-core GPU, 36 GB), macOS 15.7, PyTorch 2.15, on 300 LibriSpeech files. On an NVIDIA GPU it is much faster.

Training

• Speech: LibriSpeech train-clean-100 (English, 100 h) and Russian LibriSpeech train (93 h); validation and test speakers are disjoint.
• Degradations, generated on the fly in 8-s windows, 0–3 events per window, whole-window or 0.5–4 s intervals with random onsets, fixed signal-chain order: MUSAN noise (−5…30 dB SNR) and music (0…25 dB), RIRS_NOISES room impulse responses, hard / soft clipping (2–30 % samples), Butterworth or resampling low-pass (1–5 kHz), high-pass (200–1000 Hz), MP3 8–41 kbit/s, Opus 10–32 kbit/s, Vorbis, GSM, µ-law, A-law, G.721, IMA-ADPCM, Gilbert–Elliott packet loss (5–30 %), 50/60 Hz hum, clicks, pitch shift (±2–8 semitones), time stretch (0.7–0.9, 1.15–1.5). Labels are per 20 ms; frames where a degradation is physically absent are masked.
• Model: Whisper-small encoder, learnable weighted sum of all 13 hidden states, two 1-D convolutions and a linear head, 12 sigmoids per frame.
• Recipe: AdamW, lr 3e-5 (encoder) / 1e-3 (head), 1 000 warm-up steps, cosine, bf16, batch 64 windows, 30 000 steps (≈ 4 300 h of generated audio), one H100, 2.4 h. Best validation checkpoint at step 16 000 (frame macro-AP 0.969).

Files in this repository

file	model	size
aadd_v3_1_whisper_small_frames.pt	v3.1 (default v3): 12 realistic classes every 20 ms (this card)	177 MB
aadd_v3_whisper_small_frames.pt	v3.0: the first release of v3	177 MB
aadd_v3_1_lite_crnn_frames.pt	v3-lite: 1.3 M-parameter CRNN distilled from v3.1, same 12 classes per 20 ms; 2× faster than v3.1 on CPU but weaker on real recordings (noise ROC-AUC 0.96 on VoiceBank-DEMAND, time_stretch false alarms on 10–21 % of clean VCTK clips); Detector("v3-lite")	2.7 MB
aadd_v2_efficientnet_b0_mel.pt	v2: EfficientNet-B0 on log-mel, 7 synthetic classes per second	16 MB
aadd_v1_efficientnet_b0_png.pt	v1: EfficientNet-B0 on rendered mel images, 7 synthetic classes per second	16 MB
eval/	evaluation reports (incl. the cross-evaluation and NISQA reference runs), training logs, speed benchmark, FSD50K noise-pool clip list	—

All are loaded the same way: Detector("v1" | "v2" | "v3" | "v3.0" | "v3-lite"), or Detector(path).

v1 and v2

Earlier, smaller models for seven synthetic audiomentations degradations (Gaussian noise, pitch shift, time stretch, circular shift, clipping, low-pass 150–750 Hz, high-pass 2–8 kHz) on 1-s chunks of English read speech; trained on LibriSpeech dev-clean with one random degradation per second (p = 0.5). Inputs: v1 renders a 128-band mel spectrogram to a 310×385 PNG with matplotlib (kept bit-exact in the package), v2 uses the log-mel tensor directly (1×128×63). CPU-friendly (4 M parameters).

Regenerated LibriSpeech test-clean set, 108 936 one-second chunks, threshold 0.5:

	v1	v2
F1 macro (8 outputs)	0.831	0.919
F1 micro	0.908	0.930
exact match per second	0.906	0.927
file-level Spearman (true vs predicted number of degraded seconds)	0.932	0.938
CPU, ms per second of audio (Apple M3 Pro)	133	39

Per-class F1 (v1 / v2): Gaussian noise 1.000 / 0.999, pitch shift 0.934 / 0.912, time stretch 0.941 / 0.968, circular shift 0.000 / 0.726, clipping 0.864 / 0.818, low-pass 0.991 / 0.997, high-pass 0.992 / 0.992, none 0.926 / 0.942. These models only know their seven synthetic classes; for real recordings use v3.

Limitations

• Degradations are synthetic. Real-world tests above cover noise and reverberation; the other classes are validated only on held-out synthetic data.
• The "clean" training speech comes from LibriVox audiobooks, which themselves contain low-level noise, hum and MP3 history. The model therefore flags some of it (22 % of English and 44 % of Russian synthetic "clean" windows have a frame above 0.5 with v3.1, which is more sensitive to noise than v3.0), which caps the measured AP for noise and hum. On studio-clean speech (VoiceBank clean) the false-alarm rate is 0.0 % (DNS clean: 0.7 %).
• Read speech only; music, far-field meetings and TTS output were not evaluated.
• Inputs are resampled to 16 kHz, so content above 8 kHz is not seen.

Licence and data

Weights: MIT. Base model: Whisper-small (Apache 2.0). Training data: LibriSpeech (CC BY 4.0), Russian LibriSpeech (CC BY 4.0 card, public-domain source recordings), MUSAN (CC BY 4.0), RIRS_NOISES (Apache 2.0). Evaluation data: MIT IR survey, VoiceBank-DEMAND, DNS Challenge 2020, Dynamic-SUPERB.