Inference Speed

#8
by andreasrath - opened

Does anyone have any details on inference speeds on A100 or V100 or T4?

MetaVoice org

Hey @andreasrath , with our latest release you should expect the following

  1. A100/V100: faster than real-time inference
  2. T4: ~2x real-time today

We will be dropping support for streaming soon.

@sidroopdaska many thanks for your info!

Is there any hope for a more efficient model that can run in real-time CPUs? I used Candle to run this, even the quantized version and it took a minute for 2 seconds of audio on my AMD 7640U laptop

MetaVoice org

@soupslurpr this is not something on our roadmap for this release... feel free to find an issue on our github (https://github.com/metavoiceio/metavoice-src) and we'll be able to track it for a future release

vatsal-metavoice changed discussion status to closed

Sign up or log in to comment