AADD v3: which audio degradation, and where

Code, CLI, Python API and interactive dataset report: github.com/gertelrina/aadd.

AADD (Automatic Audio Degradation Detection) finds 12 kinds of degradation in speech recordings every 20 ms: background noise, music, reverberation, clipping, low-pass / narrow band, high-pass, codec artefacts, packet loss, mains hum, clicks, pitch shift and time stretch. Several can be present at once. It is meant for cleaning speech datasets (ASR, TTS) and for auditing recordings.

• Backbone: Whisper-small encoder (88 M parameters), fine-tuned end to end
• Output: one sigmoid per class per 20 ms frame on 8-s windows; any length via sliding windows
• Languages seen in training: English and Russian

Usage

pip install "aadd[app] @ git+https://github.com/gertelrina/aadd"

aadd scan my_dataset/ --model v3 --report report.html   # downloads this checkpoint on first use
aadd scan call.wav --model v3 --resolution 0.1 --segments per_100ms.csv

from aadd import Detector

det = Detector("v3", resolution=1.0)       # per-second summary; frame probabilities in res.frame_probs
res = det.predict_file("call.wav")
res.detected                                # {'noise': 0.97, 'reverb': 0.88}
res.frame_probs.shape                       # (n_frames_20ms, 12)

Or download the file directly:

from huggingface_hub import hf_hub_download
path = hf_hub_download("agertel/aadd", "aadd_v3_whisper_small_frames.pt")
det = Detector(path)

Results

Out-of-domain, real recordings (never seen in training). Clip score = 95th percentile of frame probabilities.

set	class	ROC-AUC	recall at p ≥ 0.5	false alarms on clean
VoiceBank-DEMAND test, 824 noisy vs 824 clean twins	noise	0.985	0.51	0.4 %
DNS Challenge 2020 synthetic dev-test, 600 clips	noise	0.997	0.85	0.0 %
DNS Challenge 2020 synthetic dev-test, 600 clips	reverberation	1.000	1.00	0.0 %
Dynamic-SUPERB noise detection (VCTK + MUSAN)	noise	0.995	0.96	1.1 %
Dynamic-SUPERB reverberation, small / medium / large room	reverberation	0.995 / 1.000 / 0.998	0.97 / 1.00 / 0.96	0 %

Ranking is near perfect; at the default threshold the model is conservative on mild noise (VoiceBank-DEMAND SNR 2.5–17.5 dB), so lower the threshold when filtering data.

Held-out synthetic test sets (unseen speakers, noises and rooms; up to three overlapping degradations with random onsets):

test set	frame macro-AP (20 ms)	1-s segment macro-F1	clip macro-AUC
English (LibriSpeech test-clean), 3 000 × 8 s	0.955	0.915	0.988
Russian (Russian LibriSpeech test), 3 000 × 8 s	0.959	0.883	0.986
English, real measured RIRs	0.951	0.909	0.989
English, MIT IR survey RIRs	0.940	0.892	0.988

Frame AP per class (English / Russian): pitch shift 0.999 / 0.992, time stretch 0.997 / 0.992, clipping 0.996 / 0.996, high-pass 0.991 / 0.998, packet loss 0.989 / 0.990, reverberation 0.987 / 0.989, music 0.983 / 0.985, clicks 0.972 / 0.975, low-pass 0.966 / 0.981, codec 0.920 / 0.949, background noise 0.885 / 0.803, mains hum 0.772 / 0.864.

Training

• Speech: LibriSpeech train-clean-100 (English, 100 h) and Russian LibriSpeech train (93 h); validation and test speakers are disjoint.
• Degradations, generated on the fly in 8-s windows, 0–3 events per window, whole-window or 0.5–4 s intervals with random onsets, fixed signal-chain order: MUSAN noise (−5…30 dB SNR) and music (0…25 dB), RIRS_NOISES room impulse responses, hard / soft clipping (2–30 % samples), Butterworth or resampling low-pass (1–5 kHz), high-pass (200–1000 Hz), MP3 8–41 kbit/s, Opus 10–32 kbit/s, Vorbis, GSM, µ-law, A-law, G.721, IMA-ADPCM, Gilbert–Elliott packet loss (5–30 %), 50/60 Hz hum, clicks, pitch shift (±2–8 semitones), time stretch (0.7–0.9, 1.15–1.5). Labels are per 20 ms; frames where a degradation is physically absent are masked.
• Model: Whisper-small encoder, learnable weighted sum of all 13 hidden states, two 1-D convolutions and a linear head, 12 sigmoids per frame.
• Recipe: AdamW, lr 3e-5 (encoder) / 1e-3 (head), 1 000 warm-up steps, cosine, bf16, batch 64 windows, 30 000 steps (≈ 4 300 h of generated audio), one H100, 2.4 h. Best validation checkpoint at step 16 000 (frame macro-AP 0.969).

Files in this repository

file	model	size
aadd_v3_whisper_small_frames.pt	v3: 12 realistic classes every 20 ms (this card)	177 MB
aadd_v2_efficientnet_b0_mel.pt	v2: EfficientNet-B0 on log-mel, 7 synthetic classes per second	16 MB
aadd_v1_efficientnet_b0_png.pt	v1: EfficientNet-B0 on rendered mel images, 7 synthetic classes per second	16 MB
eval/	evaluation reports, training logs, speed benchmark	—

All are loaded the same way: Detector("v1" | "v2" | "v3"), or Detector(path).

v1 and v2

Earlier, smaller models for seven synthetic audiomentations degradations (Gaussian noise, pitch shift, time stretch, circular shift, clipping, low-pass 150–750 Hz, high-pass 2–8 kHz) on 1-s chunks of English read speech; trained on LibriSpeech dev-clean with one random degradation per second (p = 0.5). Inputs: v1 renders a 128-band mel spectrogram to a 310×385 PNG with matplotlib (kept bit-exact in the package), v2 uses the log-mel tensor directly (1×128×63). CPU-friendly (4 M parameters).

Regenerated LibriSpeech test-clean set, 108 936 one-second chunks, threshold 0.5:

	v1	v2
F1 macro (8 outputs)	0.831	0.919
F1 micro	0.908	0.930
exact match per second	0.906	0.927
file-level Spearman (true vs predicted number of degraded seconds)	0.932	0.938
CPU, ms per second of audio (Apple M3 Pro)	133	39

Per-class F1 (v1 / v2): Gaussian noise 1.000 / 0.999, pitch shift 0.934 / 0.912, time stretch 0.941 / 0.968, circular shift 0.000 / 0.726, clipping 0.864 / 0.818, low-pass 0.991 / 0.997, high-pass 0.992 / 0.992, none 0.926 / 0.942. These models only know their seven synthetic classes; for real recordings use v3.

Limitations

• Degradations are synthetic. Real-world tests above cover noise and reverberation; the other classes are validated only on held-out synthetic data.
• The "clean" training speech comes from LibriVox audiobooks, which themselves contain low-level noise, hum and MP3 history. The model therefore flags some of it (14 % of English and 33 % of Russian synthetic "clean" windows have a frame above 0.5), which caps the measured AP for noise and hum. On studio-clean speech (VoiceBank clean) the false-alarm rate is 0.4 %.
• Read speech only; music, far-field meetings and TTS output were not evaluated.
• Inputs are resampled to 16 kHz, so content above 8 kHz is not seen.

Licence and data

Weights: MIT. Base model: Whisper-small (Apache 2.0). Training data: LibriSpeech (CC BY 4.0), Russian LibriSpeech (CC BY 4.0 card, public-domain source recordings), MUSAN (CC BY 4.0), RIRS_NOISES (Apache 2.0). Evaluation data: MIT IR survey, VoiceBank-DEMAND, DNS Challenge 2020, Dynamic-SUPERB.