ASLP-lab/WS-MIN-TTS

WenetSpeech-Min TTS Model

WenetSpeech-Min TTS is a Minnan (Southern Min) text-to-speech model based on Fun-CosyVoice3-0.5B-2512. The CosyVoice3 LLM is fine-tuned for Minnan speech synthesis, while the Flow, HiFT, speaker encoder, speech tokenizer, and text tokenizer are inherited from the base model.

Install

Clone the repo

Clone the CosyVoice repository and initialize its submodules:

git clone https://github.com/ASLP-lab/WenetSpeech-Min-Repo.git

cd CosyVoice3-Min

Create Conda env

conda create -n cosyvoice3 -y python=3.10
conda activate cosyvoice3
pip install -r requirements.txt

Model download

Download the complete checkpoint from Hugging Face:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="ASLP-lab/WS-MIN-TTS",
    local_dir="pretrained_models/WS-MIN-TTS",
)

The CosyVoice3 model files will be downloaded to pretrained_models/WS-MIN-TTS/CosyVoice3-Min/.

Usage

The following example performs zero-shot Minnan speech synthesis using a reference audio file:

import os
import sys

import soundfile as sf

sys.path.append("third_party/Matcha-TTS")

from cosyvoice.cli.cosyvoice import AutoModel


model_dir = "pretrained_models/WS-MIN-TTS/CosyVoice3-Min"
prompt_wav = "prompt.wav"
output_wav = "output.wav"

if not os.path.isfile(prompt_wav):
    raise FileNotFoundError(f"Reference audio not found: {prompt_wav}")

cosyvoice = AutoModel(model_dir=model_dir)
instruct_text = "用闽南语说这句话<|endofprompt|>"
tts_text = "你欲食啥物？"

for model_output in cosyvoice.inference_instruct2(
    tts_text,
    instruct_text,
    prompt_wav,
    stream=False,
):
    speech = model_output["tts_speech"].squeeze(0).cpu().float().numpy()
    sf.write(output_wav, speech, cosyvoice.sample_rate)

print(f"Generated audio saved to: {output_wav}")

Use a clean reference recording for prompt.wav. The generated audio is saved to output.wav.