thangylvp/vibevoice-0.5b-custom

VibeVoice 0.5B custom checkpoint

Generate a 24 kHz WAV from text with the included generate_speech.py script. Optionally supply a short reference recording to condition the voice. The matching custom inference code is included in vibevoice_custom/; the original training project and its dataset are not required.

Access

This model is publicly listed with manual approval required for downloads. Request access on this page and wait for the owner to approve it, then log in with your own Hugging Face account as shown below.

Install and download

Use Python 3.11 in a fresh virtual environment. The example was tested with PyTorch 2.8.0, Transformers 4.51.3, and Diffusers 0.37.0. Install the appropriate PyTorch 2.8.0 build for your CPU or CUDA system; the remaining dependencies are listed in requirements.txt.

python -m venv .venv
source .venv/bin/activate
python -m pip install huggingface_hub==0.36.2
hf auth login

# Download the example and bundled implementation first.
hf download thangylvp/vibevoice-0.5b-custom --local-dir vibevoice-tts --include generate_speech.py requirements.txt 'vibevoice_custom/**' SOURCE_MANIFEST.json README.md

cd vibevoice-tts
python -m pip install -r requirements.txt

The first generation downloads the 2.72 GB checkpoint and the separate Qwen/Qwen2.5-0.5B text tokenizer. Downloads are cached for subsequent runs. Authentication uses your local Hugging Face login or the HF_TOKEN environment variable; no token is included in this repository.

Text to speech

python generate_speech.py --text "Xin chào, tôi có thể giúp gì cho bạn?" --output speech.wav

This works without a reference recording. With no reference, the voice is unconditioned and its identity is not controlled.

To condition the voice on your own recording:

python generate_speech.py --text "Xin chào, đây là ví dụ chuyển văn bản thành giọng nói." --reference-audio reference.wav --output speech.wav

Use a clean 5–10 second voice recording. The script converts stereo to mono, resamples to 24 kHz, and normalizes the reference before encoding it. Recordings longer than 30 seconds are rejected. No reference recordings are distributed here.

Read longer text from a UTF-8 file and increase the speech-token limit as needed:

python generate_speech.py --text-file text.txt --max-new-tokens 500 --output speech.wav

The default 200 speech tokens permit approximately 26.7 seconds of output; generation may stop earlier at the model's end-of-speech prediction. If the script reports that the token limit was reached, raise --max-new-tokens or split the text into shorter passages.

CUDA is selected automatically when available, using BF16 on supported GPUs. Otherwise the script uses FP32. CPU inference is supported but can be slow:

python generate_speech.py --text "Xin chào." --device cpu --output speech.wav

For an already downloaded checkpoint, pass --model /path/to/checkpoint. Run python generate_speech.py --help for diffusion steps, guidance, seed, device, and dtype options. This is a custom Python implementation; loading it with a generic Transformers pipeline() or AutoModel is not supported.

Checkpoint details

• model.safetensors: 2,722,761,832 bytes; 881 BF16 tensors.
• Architecture: VibeVoiceStreaming05ForInference, vibevoice_streaming.
• Audio output: mono, 24,000 Hz, PCM-16 WAV.
• Text tokenizer: Qwen/Qwen2.5-0.5B (downloaded separately).
• Full weight file: 1,361,322,756 stored tensor elements, including speech components. The 0.5B designation is the local model variant name.

This is a Hugging Face format export of a local training checkpoint. The export workflow loads a merged checkpoint with strict state-dictionary matching, converts to BF16, and saves safetensors with the processor configuration. Training data, metrics, and the exact upstream revision have not been verified.

Validation and code provenance

Text-only and reference-conditioned generation were run on an NVIDIA RTX 5090 with the bundled implementation and the exact uploaded weights. Both produced non-empty, finite 24 kHz WAV audio. Reference preprocessing was exercised with a stereo 16 kHz recording created from the generated test audio. These are functional checks, not speech-quality or speaker-similarity benchmarks.

SOURCE_MANIFEST.json records the original inference-source checksums and adaptations. The bundle changes the original project import namespace, loads only the required components, and removes per-token debug printing. Existing source notices are preserved. No new license is assigned by this repository.