You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

F5-TTS Tensorrt-LLM Faster

为 F5-TTS 进行推理加速,测试样例如下:

  • NVIDIA GeForce RTX 3090
  • 测试文本为: 这点请您放心,估计是我的号码被标记了,请问您是沈沈吗?

经测试,推理速度由3.2s降低为0.72s, 速度提升 4 倍!

整个项目 workflow 可概述如下:

  1. 先将 F5-TTSONNX 导出,导出为三部分;
  2. 然后使用 Tensorrt-LLM 对有关 Transformer 部分进行网络的改写以便于加速,前端和decode仍使用ONNX推理,当然也可以指定CUDAExecutionProviderOpenVINOExecutionProvider等。

本仓库存储模型权重,包括 torchONNXtrtllm

详细请看(项目构建、权重转换、快速推理):https://github.com/WGS-note/F5_TTS_Faster

F5-TTS 的学习笔记也会发,敬请期待,公众号:WGS的学习笔记

开源不易,欢迎 star

特别感谢以下两个开源项目的贡献:

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model's library. Check the docs .

Model tree for wgs/F5-TTS-Faster

Base model

SWivid/F5-TTS
Quantized
(1)
this model