echo

Next Echo

Next Echo finds every moment a given example sound occurs in a longer recording β€” a clap, a knock, a bark β€” with no fixed class list. A small CNN embeds 1-second log-mel windows; the recording is scanned against the example. It is compared with plain log-mel similarity; the final score fuses both after per-recording normalisation.

Space: https://huggingface.co/spaces/thelamapi/Next-Echo β€” record a 2-second example with your microphone (or pick one), choose a recording, and listen to each hit. Runs in the browser.

Results on constructed 30 s test recordings (ESC-50 fold 5; 40 recordings per cell)

Event F1 (Β±0.5 s) / false alarms per minute β€” log-mel baseline vs Next Echo (better F1 in bold):

Classes / condition Log-mel similarity Next Echo (learned)
seen/clean 0.676 / 2.15 0.619 / 3.25
seen/noisy 0.606 / 1.45 0.630 / 2.30
seen/overlap 0.788 / 0.80 0.749 / 1.05
seen/absent – / 4.00 – / 4.95
unseen/clean 0.617 / 2.10 0.658 / 3.30
unseen/noisy 0.667 / 1.10 0.717 / 2.05
unseen/overlap 0.790 / 0.85 0.767 / 1.70
unseen/absent – / 3.65 – / 5.60
seen/all-present 0.690 / 1.47 0.662 / 2.20
unseen/all-present 0.691 / 1.35 0.712 / 2.35

unseen = the 10 classes never used in training (clapping, door_wood_knock, rooster, church_bells, glass_breaking, keyboard_typing, siren, cow, sneezing, can_opening). absent recordings contain no target, so only false alarms count. Thresholds were chosen on validation recordings built from fold 4.

Example timeline and benchmark

Method

  • Audio: 16 kHz mono, 512-point STFT (hop 256), 64 mel bands (50–8000 Hz), log.
  • Embedding: 4 conv blocks + linear β†’ 64-d, supervised contrastive loss (Ο„ = 0.1) on random 1-s crops of 960 clips from 40 classes, with gain, mixing and noise augmentation, 5000 steps.
  • Training windows mimic the test conditions: real waveform segments mixed with another sound and brown noise (0–30 dB SNR), alongside clean segments like the user's example.
  • Query: the loudest 1-s windows of the example are embedded and averaged. Recording: windows every 0.256 s.
  • Score: per-recording robust normalisation (median/MAD), then Next Echo = Ξ±Β·z(learned) + (1βˆ’Ξ±)Β·z(log-mel) with Ξ± = 0.75 chosen on validation recordings (options Ξ± ∈ {1, 0.75, 0.5}). Consecutive windows above the threshold form one detection.
  • Baseline: cosine between per-band mean/std of log-mel windows, raw or normalised β€” whichever validated better (raw).
  • The browser computes the same log-mel (own FFT) and CNN with nlrt.js; Node parity on 3 real snippets: max abs difference 7.105427357601002e-15.

Usage

python inference.py --query clap.wav --recording scene.wav     # 16 kHz mono 16-bit WAV
python inference.py --selftest

Limitations and license

  • Recordings are mixtures of real clips, not naturally recorded scenes; real rooms add reverberation and far more varied backgrounds.
  • Very short or very quiet examples give weak queries. Sounds that resemble each other (knock vs footsteps) cause false alarms.
  • Non-commercial: ESC-50 is CC BY-NC 3.0, so this model, its demo audio and the Space are for non-commercial use only.
  • Trained on NVIDIA L4; profile full. Failure cases are listed in evaluation.json.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Space using thelamapi/Next-Echo 1

Collection including thelamapi/Next-Echo