malerlab/unirqvae-ytsv

U-MusT score-image codec (RQ-VAE, earlier generation)

The earlier generation of the score-image tokenizer used by U-MusT. Same architecture as unirqvae3-ytsv — 16x compression, 4 codebooks of 1024 entries, attention blocks removed — trained on a different data selection.

Use unirqvae3-ytsv unless you specifically need this generation. Every image-token dataset released alongside the paper is tokenized with unirqvae3. This codec is provided because the released strings translation model was trained against it, and image tokens are not interchangeable between codec generations.

Usage

vq_models/unirqvae_f16_c1024_k4/config.yaml
vq_models/unirqvae_f16_c1024_k4/epoch41_model.pt

Licensing

This release is licensed CC BY-NC-SA 4.0.

It is derived from a training mixture whose corpora carry differing terms: MAESTRO, ASAP and MusicNetEM (CC BY-NC-SA 4.0), MusicNet and SLakh (CC BY 4.0), BPSD (CC BY 3.0), OLiMPiC (CC BY-SA 4.0), and YTSV metadata (CC BY-NC-SA 4.0). The non-commercial and share-alike terms are applied here because they are the most restrictive verified terms in that chain.

One conflict is irreducible and is disclosed rather than resolved: CC BY-SA 4.0 and CC BY-NC-SA 4.0 are mutually incompatible, since the former permits commercial use and forbids adding restrictions while the latter requires them. Any work derived from both — including these weights — cannot satisfy both simultaneously. See the code repository for the full per-corpus license table and the conflicts.

This is not legal advice. Verify the current terms with each corpus publisher before redistributing anything derived from this release or using it commercially.

Citation

@article{jung2026umust,
  title   = {U-MusT: A Unified Framework for Cross-Modal Translation of Score Images, Symbolic Music, and Performance Audio},
  author  = {Jung, Jongmin and Kim, Dongmin and Lee, Sihun and Cho, Seola and Soh, Hyungjoon and Bukey, Irmak and Donahue, Chris and Jeong, Dasaem},
  journal = {IEEE Transactions on Audio, Speech and Language Processing},
  volume  = {34},
  pages   = {1876--1891},
  year    = {2026},
  doi     = {10.1109/TASLPRO.2025.3648794}
}