CosyVoice2-0.5B

CosyVoice 2.0 是基于大语言模型（LLM）的可流式语音合成（TTS）系统，支持多语言、零样本语音复刻、跨语种复刻与自然语言控制。

• 官方仓库：FunAudioLLM/CosyVoice
• 原始模型：FunAudioLLM/CosyVoice2-0.5B
• 论文：CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models

特性

• 零样本语音复刻（3 秒参考音频即可克隆音色）
• 跨语种 / 跨方言合成
• 自然语言控制（instruct 控制语速、情绪、方言等）
• 流式推理，延迟低至 150ms

使用方式

需要配合 CosyVoice 官方代码使用。

git clone --recursive https://github.com/FunAudioLLM/CosyVoice.git
cd CosyVoice
pip install -r requirements.txt

下载本模型：

from modelscope import snapshot_download
snapshot_download('taoyy/CosyVoice2-0.5B', local_dir='pretrained_models/CosyVoice2-0.5B')

或使用 Hugging Face：

from huggingface_hub import snapshot_download
snapshot_download('taoyy/CosyVoice2-0.5B', local_dir='pretrained_models/CosyVoice2-0.5B')

推理示例：

import sys
sys.path.append('third_party/Matcha-TTS')
from cosyvoice.cli.cosyvoice import AutoModel
import torchaudio

cosyvoice = AutoModel(model_dir='pretrained_models/CosyVoice2-0.5B')

# 零样本复刻
for i, j in enumerate(cosyvoice.inference_zero_shot(
        '收到好友从远方寄来的生日礼物，那份意外的惊喜与深深的祝福让我心中充满了甜蜜的快乐。',
        '希望你以后能够做的比我还好呦。',
        './asset/zero_shot_prompt.wav')):
    torchaudio.save('zero_shot_{}.wav'.format(i), j['tts_speech'], cosyvoice.sample_rate)

引用

@article{du2024cosyvoice,
  title={Cosyvoice 2: Scalable streaming speech synthesis with large language models},
  author={Du, Zhihao and Wang, Yuxuan and Chen, Qian and Shi, Xian and Lv, Xiang and Zhao, Tianyu and Gao, Zhifu and Yang, Yexin and Gao, Changfeng and Wang, Hui and others},
  journal={arXiv preprint arXiv:2412.10117},
  year={2024}
}

免责声明

本模型仅用于学术研究与技术演示。