MAGDA Sample Tagger

ONNX exports of LAION's CLAP HTSAT-unfused model plus the RoBERTa tokenizer, packaged for the MAGDA DAW's sample library (issue #768).

What's in this repo

clap_audio.onnx — audio encoder. Takes a mono 48 kHz waveform, produces a 512-d normalised embedding suitable for cosine similarity search.
clap_text.onnx — text encoder. Takes RoBERTa token ids + attention mask, produces a 512-d normalised embedding in the same space as the audio encoder so a text query can rank audio files by similarity.
tokenizer.json — the RoBERTa BPE tokenizer that pairs with the text encoder. MAGDA's C++ tokenizer reads this file directly.

MAGDA's media database (a SQLite catalogue of audio samples) uses these encoders to:

Compute an embedding per indexed sample at index time, stored in the media_embedding table.
Encode the user's free-text search query at query time and rank samples by cosine similarity to the query embedding.

Without these models MAGDA falls back to filename / tag full-text search — still useful, just no semantic similarity.

ONNX exports are generated from laion/clap-htsat-unfused via the export script in MAGDA's prototype:

prototypes/media_db/src/media_db/embeddings/onnx_export.py

Notes:

Run on CPU (MPS does not support float64 used by the audio encoder's mel filterbank).
Requires transformers >= 5.x. The audio-feature accessor was renamed from audios= to audio= between 4.x and 5.x; passing the old kwarg silently returns wrong shapes.
tokenizer.json is the unmodified file from the upstream HF repo, fetched via AutoTokenizer.from_pretrained(...).save_pretrained(...).

BSD-3-Clause — same as the upstream LAION CLAP weights. See the upstream repo for the original notice and attribution.

Downloads last month: -; Downloads are not tracked for this model. How to track