HeartCodec-oss-fp16 (MLX)

The decoder half of HeartCodec-oss at float16, for mlx-heartmula-swift. It turns HeartMuLa's 8 × 8192 RVQ codes (12.5 Hz) into 48 kHz stereo: a flow-matching LLaMA-DiT (24 × 1536 + 6 × 3072, 10 Euler steps, CFG 1.25) to 25 Hz latents in 29.76 s windows, then the SQ-Codec decoder (1920× upsampling). This is the default codec: half the size of the fp32 codec and 3.5× faster per window.

file	size
model.safetensors (506 tensors, float16; decode path only, weight_norm folded, MLX conv layouts)	3.15 GB
config.json (with an mlx_heartmula section)	1.9 KB
LICENSE (Apache-2.0), NOTICE	—

Never bf16: upstream says bf16 degrades the codec's audio.

Use

This codec is what HeartMuLaConfiguration picks unless told otherwise. It works with either LM repo.

// Package.swift: .package(url: "https://github.com/xocialize/mlx-heartmula-swift", from: "0.1.1") → product "MLXHeartMuLa"
import MLXHeartMuLa
import MLXServeCore
import MLXToolKit

let engine = MLXServeEngine()
await engine.useModelStore(ModelStore(root: modelsRoot))
let id = try await engine.register(HeartMuLaPackage.registration,
                                   configuration: HeartMuLaConfiguration(),   // .int8 LM + .fp16 codec
                                   id: "heartmula-oss-3b")
try await engine.prepare(.music, package: id)    // mlx-community/HeartMuLa-oss-3B-8bit + this repo

TF32 doesn't matter here. With the fp16 codec, a song is byte-identical whether MLX's TF32 matmuls are on or off, so it loads in a process where another model already ran. On the same codes it sits 23.6 dB from the fp32 codec (the SQ-Codec's rounding step amplifies fp16 rounding), with lyric and aesthetic scores unchanged.

LM + codec	declared memory (resident + peak working set)	fits	render speed (M5 Max)
8-bit + fp16 (the default)	8.5 + 5.5 GB	24 GB Mac	3.0× real time
bf16 + fp16	11.5 + 6.0 GB	32 GB Mac	2.5×
8-bit + fp32	11.5 + 10.0 GB	32 GB Mac	2.3×
bf16 + fp32	14.5 + 10.0 GB	48 GB Mac	1.85×

Memory is phys_footprint through MLXEngine at a 120 s song and a 240 s worst case; "fits" is the engine's 0.7 × RAM admission rule. A 4-bit LM failed its lyric gate (PER 0.220) and is not published.

Measured character

Read on the fleet's 12-song set (10 English, 2 Mandarin; two seed sets per tier) and in the operator's listening passes:

• Lyrics are followed closely in English. The lyric error rate (Whisper PER) is 0.125 with the bf16 LM and 0.132 with the 8-bit LM (the PyTorch reference: 0.165).
• Style tags steer it weakly. CLAP style retrieval ranks its songs about 4.5 of 12 for their own tags, against about 2 for the other open music models measured. A higher CFG (3.0) does not help.
• Voice gender drifts. Most voices come out female whatever the tags ask: 1 of 6 male-voice prompts got a male voice.
• Mandarin is weak and uneven. The PyTorch reference read PER 0.457 in Mandarin (one verse lost) against 0.107 in English. The Swift bf16 takes did better on the two Mandarin songs, which is too few to call it fixed.
• The 8-bit LM runs to the length cap twice as often as bf16 (10 of 24 songs against 5 of 24 at a 120 s cap). The lyrics are sung, then choruses repeat, and one song in 24 looped a single line until the cap.
• Raw codec output can exceed full scale (peaks to 1.37), so the Swift pipeline applies a single −1 dBFS limiter gain.

Parity

Every tensor is the release's own float32 value (weight_norm folded in float32) cast to float16. The fp32 codec it derives from matches upstream heartlib at 115–118 dB end to end. Details: MEASUREMENTS.md.

Licence and attribution

Apache-2.0. HeartMuLa's generator repositories carry license: apache-2.0 on their model cards but no LICENSE file. The grant is the card statement plus heartlib's README of 2026-01-20, which moves "all related model weights" to Apache-2.0. This repo ships heartlib's Apache-2.0 text as LICENSE, and a NOTICE with that grant chain and exactly what was changed. Not endorsed by the HeartMuLa team or MuLa Labs. Paper: arXiv 2601.10547.