BarkVTrace model weights
Weights for BarkVTrace, the pipeline of the master's thesis BarkVTrace: Audio-Visual Attribution of Dog Vocalizations in Real-World Videos. For every dog sound in a video it answers which dog made it, or "outside" when the vocalizing dog is off screen.
The repository's download_models.sh fetches these files together with the public models into models/ and
third_party/beats_sed/; there is no need to download them by hand.
| Path | What | Used by |
|---|---|---|
synchformer/dog_ft_20261026/seed{0,1,2}.pt |
Dog fine-tuned Synchformer heads ("this crop made the sound"), 3 seeds | Barker attribution |
lab_beats_ft/final_vocal_noise.pt |
Noise-robust vocalization BEATs head | Dog sound detection |
lab_beats_ft/final_dog.pt |
All-dog-sound BEATs head | Dog sound detection |
dog_sound_types/classifier.joblib |
Sound-type classifier (bark, growl, grunt, howl, moan, pant, whine) | Sound types |
beats_sed/ |
The lab's BEATs sound-event-detection model: beats_work/desed_task code, hparams.yaml, epoch=109-step=6160.ckpt, and Microsoft's BEATS_iter3_plus_AS2M.pt |
Base of the two BEATs heads |
Results (from the thesis)
| Stage | Test | Result |
|---|---|---|
| Detection | 45 videos (110 labeled dog sounds, 31 videos without any) | 86.0% precision, 87.9% recall |
| Barker | 297 labeled sounds, held out by YouTube channel | 75.1% right, 2.7% wrong dog |
| End to end | 106 sounds in 14 videos | 71.7% right, 0% wrong dog |
License and credits
MIT. beats_sed/ is published with the permission of the lab that trained it. BEATS_iter3_plus_AS2M.pt is
BEATs by Microsoft (MIT); the desed_task code builds on the
DCASE task 4 baseline (MIT). Synchformer (MIT) is by Iashin et al.; its
base checkpoint is downloaded from the original release.
Note: classifier.joblib and the .pt / .ckpt files are pickles; load them only from this repository.