Instructions to use billchen42/npu-cosyvoice with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- CosyVoice
How to use billchen42/npu-cosyvoice with CosyVoice:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use billchen42/npu-cosyvoice with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf billchen42/npu-cosyvoice:Q4_0 # Run inference directly in the terminal: llama cli -hf billchen42/npu-cosyvoice:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf billchen42/npu-cosyvoice:Q4_0 # Run inference directly in the terminal: llama cli -hf billchen42/npu-cosyvoice:Q4_0
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf billchen42/npu-cosyvoice:Q4_0 # Run inference directly in the terminal: ./llama-cli -hf billchen42/npu-cosyvoice:Q4_0
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf billchen42/npu-cosyvoice:Q4_0 # Run inference directly in the terminal: ./build/bin/llama-cli -hf billchen42/npu-cosyvoice:Q4_0
Use Docker
docker model run hf.co/billchen42/npu-cosyvoice:Q4_0
- LM Studio
- Jan
- Ollama
How to use billchen42/npu-cosyvoice with Ollama:
ollama run hf.co/billchen42/npu-cosyvoice:Q4_0
- Unsloth Desktop
- Docker Model Runner
How to use billchen42/npu-cosyvoice with Docker Model Runner:
docker model run hf.co/billchen42/npu-cosyvoice:Q4_0
- Lemonade
How to use billchen42/npu-cosyvoice with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull billchen42/npu-cosyvoice:Q4_0
Run and chat with the model
lemonade run user.npu-cosyvoice-Q4_0
List all available models
lemonade list
- Atomic Chat
npu-cosyvoice prebuilt files
Precomputed artifacts for npu-cosyvoice, which runs
CosyVoice3's flow-matching DiT on the AMD Ryzen AI (XDNA1) NPU. The voice bundle's setup step
downloads these instead of exporting them locally, which saves about 15 minutes of CPU time.
| path | what | made by |
|---|---|---|
weights/ |
int8 DiT weights (W8A8, per-output-channel scales, SmoothQuant on q/k/v) and AdaLN weights | python -m npu_cosyvoice.export |
cv_llm_q4_0.gguf |
CosyVoice3's Qwen2 LM backbone (no output head), Q4_0, for llama.cpp | python -m npu_cosyvoice.llm (llama.cpp 9adc7f4) |
aot/npu1/ |
the 512x1024x1024 int8 GEMM overlay for XDNA1 | scripts/compile_xclbin.py --device npu1 |
SHA256SUMS |
checksums; setup verifies them |
All three are deterministic: rebuilding them with the repository's container reproduces these files
bit for bit (see the repository README). You still need the original CosyVoice3 checkpoint for the
parts that stay in PyTorch (vocoder, embeddings, voice encoder); setup downloads it from ModelScope.
License and attribution
Derived from Fun-CosyVoice3-0.5B-2512
by FunAudioLLM, released under Apache-2.0; these files are distributed under the same license
(see LICENSE). Modifications: the DiT projection weights were quantized to int8 with SmoothQuant
scaling on the q/k/v input, calibrated on four sentences; the LM backbone was converted to GGUF and
quantized to Q4_0 with its unused output head removed. The NPU overlay is generated with
MLIR-AIE (Apache-2.0 WITH LLVM-exception).
- Downloads last month
- -
4-bit
Model tree for billchen42/npu-cosyvoice
Base model
FunAudioLLM/Fun-CosyVoice3-0.5B-2512