Instructions to use microsoft/VibeVoice-ASR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/VibeVoice-ASR with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="microsoft/VibeVoice-ASR")# Load model directly from transformers import VibeVoiceForASRTraining model = VibeVoiceForASRTraining.from_pretrained("microsoft/VibeVoice-ASR", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Hosted API for VibeVoice-ASR
Noticed there's no provider actually serving this model, so I put one up: https://vibevoice-asr.com
Upload audio, get back the structured speaker/timestamp/text output.
Happy to hear what's missing.
Good of you to stand one up. The thing I would want next is streaming. Upload-and-return covers transcription after the fact, but anything interactive needs partial results arriving while the audio is still going, which is a different shape of API rather than a faster version of this one.
After that, word-level timestamps with a confidence value, and some way to bias the vocabulary toward names and product terms. That last one is where transcription output usually goes wrong in practice, and it is rarely the general accuracy that lets you down.
Thanks, this is really useful feedback.
Streaming output and hotwords are both on the list β hotwords especially, since the model already supports them and I just haven't exposed the field yet. Word-level timestamps and confidence too.
The one I can't do is streaming in and out. The model needs the full audio encoded before it starts decoding, and the speaker assignment depends on having seen the whole thing. It's architectural, not something I can work around.
What are you building? If you need live partials, a different backend would serve you better and I'd rather say that than have you find out later.