schism-audio/ast-audioset-10-10-coreml

AST (AudioSet, 10-10) — Core ML

MIT/ast-finetuned-audioset-10-10-0.4593 (Audio Spectrogram Transformer, 527-class AudioSet tagging) for Core ML on Apple devices.

Converted from the same verified reference used by the schism-mlx MLX ports. Two variants per model:

File	Precision	Compute units	max logit diff
AST_fp16.mlpackage	FLOAT16	ALL (ANE)	7.6e-3, top-5 identical on tested clips
AST_fp32.mlpackage	FLOAT32	CPU+GPU	1.6e-5

Verified on-device-equivalently via coremltools on an M5 Max, against the reference implementation on real audio. fp16 is ANE-eligible and recommended for iPhone / iPad; fp32 is the tight-parity fallback.

Download

.mlpackage bundles must be materialized as real files — the Core ML compiler rejects the symlinks that a default snapshot_download creates in the Hugging Face cache:

from huggingface_hub import snapshot_download
path = snapshot_download("schism-audio/ast-audioset-10-10-coreml", local_dir="./ast-audioset-10-10-coreml")

(or hf download schism-audio/ast-audioset-10-10-coreml --local-dir ./ast-audioset-10-10-coreml). Swift hosts downloading files directly are unaffected.

I/O contract

• input features: (1, 1024, 128) float32 — Kaldi-style fbank (16 kHz, 128 mels), zero-padded/truncated to 1024 frames, normalized (x + 4.2677393) / (2 * 4.5689974)
• output logits: (1, 527) float32 — apply sigmoid; multi-label
• longer audio: slice the full-file fbank into 1024-frame windows every 512 frames plus a final end-aligned window, aggregate sigmoid scores per class (max or mean) — pinned by test_vectors_pipeline.npz; short audio pads the raw fbank then normalizes (padding rows ≈ 0.467, not 0)

DSP frontend (host-side)

The Core ML graph contains the network only. The host implements the audio frontend and must match schism_mlx.audio numerically — test_vectors_*.npz in this repo holds deterministic input/output pairs (float32; match within ~1e-4 relative to be interchangeable with what these models were verified against). test_vectors_pipeline.npz additionally pins the full-track orchestration around the model call (chunk grid, blending, normalization) via a deterministic mock model — scaled-down sizes, same algorithm, generated by the shipped schism-mlx separate() / windowing code. test_vectors_integration.npz goes one step further: a full synthetic track plus the verified reference output, for end-to-end host validation (the schism-dsp integration tests run the published fp32 core through it). A validated Swift implementation (Accelerate; modules SchismDSP and SchismPipeline) is available at schism-audio/schism-dsp, tested against these exact vectors.

License

BSD-3-Clause, inherited from the source checkpoint (MIT, original repo). Core ML conversion by schism-audio.