D-MaN77/HeartCodec-oss-encoder

HeartCodec OSS Encoder

Encoder-only weights for HeartCodec audio tokenization. Pair this checkpoint with HeartCodec-oss-20260123, which supplies the shared RVQ quantizer and audio decoder. Together they support audio → eight token streams at 12.5 Hz → 48 kHz stereo reconstruction.

The matching code and reconstruction examples are in HeartMuLa/heartlib.

We hope this contribution will be useful to the music research community.

Installation and reconstruction

Use Python 3.10, a CUDA-compatible PyTorch installation, and ffmpeg for MP3 output. Install heartlib in a separate environment.

git clone https://github.com/HeartMuLa/heartlib.git
cd heartlib
pip install -e .
hf download HeartMuLa/HeartCodec-oss-encoder --local-dir ./ckpt/HeartCodec-oss-encoder
hf download HeartMuLa/HeartCodec-oss-20260123 --local-dir ./ckpt/HeartCodec-oss-20260123
python examples/run_music_reconstruction.py \
  --encoder_path ./ckpt/HeartCodec-oss-encoder \
  --decoder_path ./ckpt/HeartCodec-oss-20260123 \
  --input_path ./assets/reference.mp3 \
  --save_path ./reconstructed.mp3

Replace --input_path to use your own authorized audio. MP3 is encoded directly from float32 output at 320 kbps, without an intermediate PCM WAV. WAV output is available by selecting a .wav output path. Output is trimmed to the input duration.

Python API

import torch
from heartlib.heartcodec.modeling_heartcodec import HeartCodec

codec = HeartCodec.from_encoder_decoder_pretrained(
    "HeartMuLa/HeartCodec-oss-20260123",
    "HeartMuLa/HeartCodec-oss-encoder",
    dtype=torch.float32,
).to("cuda").eval()

# waveform: float32 tensor [samples], [1, samples], or [2, samples]
# sample_rate: positive integer sampling rate of the input waveform
# tokens = codec.tokenize(waveform, sample_rate, batch_size=1)
# reconstructed = codec.detokenize(tokens, num_steps=10, guidance_scale=1.25)

Do not pass this encoder-only repository to HeartCodec.from_pretrained(). The existing decoder-only loading API remains supported. Encoder and decoder weights are strictly checked by the split-checkpoint loader.

Files

• encoder.safetensors: 1,006 encoder tensors, approximately 2.15 GB; original encoder.* names, shapes, dtypes and values are preserved.
• encoder_config.json: feature extractor and query encoder configuration.
• SHA256SUMS: weight-file SHA-256 checksum.
• LICENSE: the source checkpoint's Apache-2.0 license.

No decoder or RVQ weights are included here. The shared quantizer belongs to the paired decoder checkpoint; loading two independently trained quantizers is not supported.

Validation

On an NVIDIA B300, the split encoder and published decoder completed a 239.293-second stereo audio round trip using FP32, batch size 1, seed 42, 10 decoder steps and guidance scale 1.25. Tokens had shape [8, 2992]. Output was finite, non-silent, 48 kHz stereo and matched the input duration. PyTorch peak allocated memory was approximately 12.65 GiB; peak reserved memory was approximately 15.41 GiB. These are measurements for one input and configuration, not minimum-memory requirements or an audio-quality benchmark.

All exported encoder tensor bytes match the source checkpoint. The paired decoder's 818 tensors match the source decoder numerically; eight RVQ initialization flags differ only in stored shape (scalar versus [1]).

License

Apache-2.0, as declared by the source checkpoint. See LICENSE.