abliter8-ai/Roo-Voice_MOSS_TTS_LT_GGUF

Roo-Voice · MOSS-TTS-Local-Transformer · GGUF (llama.cpp)

Roo's voice — that signature baritone with the estuary accent that stands the hair up on the back of your neck — as a GGUF model for llama.cpp. This is the cross-vendor build: one file that runs on AMD, NVIDIA, and CPU (Vulkan / ROCm / CUDA / Metal), because llama.cpp is portable across all of them. Load it, and Roo can whisper to you all day long.

A GGUF conversion of a full-model supervised fine-tune of MOSS-TTS-Local-Transformer, trained on Roo's own recordings and specialised on his single voice.

⚠️ What actually makes the voice — read first

This is a reference-conditioned model, and both halves matter:

• The fine-tune is what makes it Roo. The base model has never heard this speaker — a base model plus any reference clip will not give you Roo's baritone or his estuary accent. That voice lives in the weights, put there by the supervised fine-tune on his recordings.
• The reference completes the delivery. reference_24k.wav (bundled) conditions the fine-tuned model at inference and is required to produce the voice.

Files

File	What	Size
moss-tts-local-roo-ip172-Q4_K_M.gguf	4-bit (recommended) — smallest, GPU-friendly	~1.76 GB
moss-tts-local-roo-ip172-BF16.gguf	Full-precision GGUF (largest quality headroom)	~5.5 GB
reference_24k.wav	The identity reference (required)	—

Why GGUF here — the cross-vendor lane

The same portable model runs on both GPU vendors; only a couple of formats are locked to one:

Build	AMD	NVIDIA	Apple
This GGUF (llama.cpp)	✅ Vulkan / ROCm	✅ CUDA / Vulkan	✅ Metal
MLX 4-bit / 8-bit	—	—	✅ only
int4 / int8 (bitsandbytes)	⚠️ experimental ROCm	✅	—

If you're on AMD Radeon, this GGUF is your path — bitsandbytes (the int4/int8 builds) is NVIDIA-first and unreliable on consumer Radeon, so use llama.cpp's Vulkan or ROCm backend here instead.

AMD support is experimental / best-effort. This build is validated on NVIDIA CUDA. On AMD, a desktop-class Radeon (RDNA3/4 dGPU) with ROCm is expected to accelerate well, but we could not fully validate it on our own hardware. Note: on a low-power iGPU (e.g. Radeon 890M) the current MOSS-TTS llama.cpp fork's custom ops did not offload to the GPU under Vulkan in our testing and ran CPU-bound — so treat iGPU performance as CPU-class for now. Contributions of Vulkan/ROCm kernels for the MOSS-TTS ops would improve this. For AMD, SGLang-Omni / vLLM-Omni (ROCm) are also worth trying for the transformers builds.

How to run it

MOSS-TTS Local is a two-part system: an LM backbone (this GGUF, run by llama.cpp) and a neural audio codec (the MOSS-Audio-Tokenizer, run via ONNX Runtime) that turns the model's tokens into 24 kHz audio and encodes the reference. You need the OpenMOSS llama.cpp integration, which adds the llama-moss-tts tool and the delay-pattern / 32-codebook decoding:

1. Build the fork — OpenMOSS's MOSS-TTS llama.cpp integration (PR). Build with your GPU backend: -DGGML_VULKAN=ON (portable, best on AMD), -DGGML_CUDA=ON (NVIDIA), or -DGGML_HIP=ON (AMD ROCm).
2. Get the codec — OpenMOSS-Team/MOSS-Audio-Tokenizer (Apache-2.0). Export or fetch its ONNX encoder/decoder for ONNX Runtime. Install onnxruntime (CPU) or the GPU package matching your CUDA/ROCm for fast decode.
3. Build the generation reference from reference_24k.wav + your text (tools/tts/moss-tts-build-generation-ref.py), then run llama-moss-tts with --n-gpu-layers -1 and point --audio-decoder-onnx at the codec. It emits a 24 kHz WAV.

Note on speed: put the codec on the GPU (an onnxruntime GPU EP, or DirectML on Windows/AMD). The LM backbone is fast on GPU; if the codec decode runs on CPU it becomes the bottleneck.

Decoding contract for this voice: seed 42, temperature 1.0, top-k 50, top-p 0.95, repetition penalty 1.1, 32 RVQ codebooks, 24 kHz.

Prefer a turnkey path?

If you don't want to build the fork: on Apple Silicon use the MLX builds; on NVIDIA use the int4 / int8 transformers builds. The GGUF is the one to reach for when you want AMD or a single cross-vendor file.

Limitations

• Reference-conditioned — the bundled reference_24k.wav must ride along; there is no text-only path.
• Single voice by design (this is Roo, not a multi-speaker system).
• Requires the OpenMOSS MOSS-TTS llama.cpp integration + the ONNX audio codec — it is not a drop-in llama-cli model.

Provenance & license

Quantized/exported form of an accepted single-speaker MOSS-TTS Local supervised fine-tune. The base model and audio codec are Apache-2.0 (OpenMOSS); weights derived from them are redistributed here under the same license.