Instructions to use Aero-Ex/ComfyUI-Vibevoice with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- VibeVoice
How to use Aero-Ex/ComfyUI-Vibevoice with VibeVoice:
import torch, soundfile as sf, librosa, numpy as np from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference # Load voice sample (should be 24kHz mono) voice, sr = sf.read("path/to/voice_sample.wav") if voice.ndim > 1: voice = voice.mean(axis=1) if sr != 24000: voice = librosa.resample(voice, sr, 24000) processor = VibeVoiceProcessor.from_pretrained("Aero-Ex/ComfyUI-Vibevoice") model = VibeVoiceForConditionalGenerationInference.from_pretrained( "Aero-Ex/ComfyUI-Vibevoice", torch_dtype=torch.bfloat16 ).to("cuda").eval() model.set_ddpm_inference_steps(5) inputs = processor(text=["Speaker 0: Hello!\nSpeaker 1: Hi there!"], voice_samples=[[voice]], return_tensors="pt") audio = model.generate(**inputs, cfg_scale=1.3, tokenizer=processor.tokenizer).speech_outputs[0] sf.write("output.wav", audio.cpu().numpy().squeeze(), 24000) - Notebooks
- Google Colab
- Kaggle
VibeVoice 7B for ComfyUI
Single-file checkpoints of Microsoft VibeVoice 7B (VibeVoice-Large) for the
native ComfyUI node pack Aero-Ex-ComfyUI-Vibevoice.
They do not load in the original vibevoice package or in transformers.
| File | Size | Notes |
|---|---|---|
vibevoice_7b_bf16.safetensors |
17.6 GB | Full precision reference. |
vibevoice_7b_int8_convrot.safetensors |
10.5 GB | INT8 ConvRot (per-channel, group 256) weights. Recommended for GPUs under 16 GB. |
Put one in ComfyUI/models/text_encoders/ and load it with Load VibeVoice Model.
Conversion
The weights are the original VibeVoice-Large weights, repacked into one file with the tokenizer and config:
- attention Q/K/V and MLP gate/up projections merged,
- LM head pruned to the four speech control tokens,
- INT8 file quantized with comfy-kitchen's TensorWiseINT8 ConvRot layout from the bf16 file.
Checked against the transformers VibeVoice implementation: identical prompt and generated speech tokens with matched noise.
Usage
Speaker 1: Welcome back to the podcast.
Speaker 2: Thanks, it is great to be here.
Connect one 5–15 s voice sample per speaker. The model was trained on up to 4 speakers.
License and responsible use
MIT, as the original release. VibeVoice can clone voices: only use voices you have permission to use, and disclose AI-generated audio. See Microsoft's model card for the original risks and limitations.
Model tree for Aero-Ex/ComfyUI-Vibevoice
Base model
aoi-ot/VibeVoice-Large