chen-jan/dcsae-musicgen

DC-SAE on MusicGen-large, layer 24

Dual-contrastive sparse autoencoders from Dual-Contrastive Sparse Autoencoders Reveal Features of Musical Interpretation (NeurIPS 2026, OpenReview). Each model splits MusicGen-large, layer 24's activations (2048 numbers per position) into two sparse codes of 32,768 features each, 50 active per position: z_i for which work is being played and z_j for how it is played.

Folder	Corpus	Checkpoint
classical/	Classical: 100 works x 10 YouTube recordings	step 15000
jazz/	Jazz standards: 100 works x 10 recordings	step 15000
pop/	Pop covers (SHS100K): 100 works x 10 recordings	step 15000

Each folder has model.safetensors (float32) and model.json (sizes, corpus, step, and the sha256 of the paper's original checkpoint it was exported from). These are the exact weights behind the paper's numbers.

labels.jsonl holds the paper's feature labels: one row per retained z_j feature (on in 1-25% of windows), with a 5-10 word label, summary, confidence and caveats, the 10 top windows and 3 control windows (YouTube video id + start/end seconds), the feature's steering strength (top10_token_peak_mean), and two flags: concept_type (pitch/harmony, generic note, non-note, ambiguous) and named_piece. z_i features are not labelled. The labels were written automatically (Music Flamingo captions, then Gemini 3.5 Flash) and not human-audited: about two thirds name a pitch class, and named-piece labels are often wrong, so filter with the flags. The control windows of this label set were not chosen by MERT similarity; the released dcsae-label does that.

Load with the dcsae package (code release coming with the camera-ready):

from huggingface_hub import hf_hub_download
from dcsae.model import load_model
path = hf_hub_download("chen-jan/dcsae-musicgen", "jazz/model.safetensors")
hf_hub_download("chen-jan/dcsae-musicgen", "jazz/model.json")
model = load_model(path)

License: CC-BY-NC 4.0 (non-commercial), because MusicGen-large's weights are CC-BY-NC 4.0 and the labels are made from Music Flamingo captions (NVIDIA's non-commercial license).