Harsha001/Drama-3-ROCmFPX-GGUF

Drama 3 (35B-A3B) — ROCmFPX GGUF

Built with Fish Audio.

Fish Audio Drama 3 (35B parameters, 3B active; speech in 83 languages, one speaker or a scene of up to 16, with cues such as [laughs] in the text) quantized to ROCmFPX weight types for speech.cpp. Each file holds the whole model, its audio decoder and its 44.1 kHz codec.

These files run only in speech.cpp built against the ROCmFPX fork of ggml, with the patch in this repo (speechcpp-rocmfpx.patch). llama.cpp, LM Studio, Ollama and stock speech.cpp cannot run them.

Not made, sponsored or endorsed by Fish Audio. Use is subject to the Fish Audio Community License 1.0 (LICENSE, NOTICE): free for research and non-commercial use, commercial use only below USD 500,000 yearly revenue, and no hosted speech or voice-cloning service without a separate agreement. Do not imitate a real person's voice without their consent.

Files and measured quality

Each file spoke the same 20 English sentences (about 100 s of speech) with --voice none at seeds 1, 2 and 3. Qwen3-ASR 1.7B transcribed the audio, and CER / WER are the character and word error rates of that transcript against the text (lower is better; mean of the three seeds). The reference is the F16 file (the F32 file does not fit in the test machine's memory). The same file moves by about a point from seed to seed, so differences of a point or so between files are noise. Some errors are only spelling, such as "7:45" for "seven forty-five", and count the same for every file.

File	Size	CER	WER	CER per seed	Speed on the test CPU	Notes
F16 (reference, not published)	71.6 GB	2.25%	2.09%	3.61 / 1.25 / 1.90	0.25x realtime
Drama-35B-A3B-v3-Q8_0_ROCMFPX.gguf	37.4 GB	3.69%	3.69%	6.82 / 2.03 / 2.23	0.39x realtime	ROCmFPX 8-bit (UE4M3 block scales). Here Q4_0_ROCMI4 is no larger and mis-spoke less
Drama-35B-A3B-v3-Q6_K_ROCMFPX.gguf	29.6 GB	2.45%	3.20%	3.94 / 1.90 / 1.51	0.03x realtime	ROCmFPX 6-bit (preview format). Here Q4_0_ROCMI4 is no larger and mis-spoke less
Drama-35B-A3B-v3-Q4_0_ROCMFP4.gguf	20.8 GB	1.92%	2.21%	0.98 / 2.56 / 2.23	0.11x realtime	ROCmFP4 4-bit, recommended for AMD GPUs. Here Q4_0_ROCMI4 is no larger and mis-spoke less
Drama-35B-A3B-v3-Q4_0_ROCMFP4_FAST.gguf	19.7 GB	3.15%	3.94%	4.53 / 0.66 / 4.27	0.08x realtime	ROCmFP4 4-bit, one scale per block (speed layout). Here Q4_0_ROCMI4 is no larger and mis-spoke less
Drama-35B-A3B-v3-Q4_0_ROCMI4.gguf	19.7 GB	1.88%	2.09%	1.90 / 2.69 / 1.05	0.06x realtime	signed 4-bit integers with UE4M3 scale
Q8_0 (stock ggml, baseline only)	38.5 GB	4.35%	4.43%	4.20 / 5.31 / 3.54	0.39x realtime
Q4_K (stock ggml, baseline only)	20.8 GB	2.12%	2.22%	1.51 / 3.28 / 1.57	0.46x realtime

The 6-bit file is named Drama-35B-A3B-v3-Q6_K_ROCMFPX.gguf so that Hugging Face lists it as a 6-bit quant. Its weights are the ROCmFPX Q6_0_ROCMFPX type, not a stock llama.cpp Q6_K, and it needs the same fork as the other files.

Honest comparison: Q8_0_ROCMFPX mis-spoke more often than stock Q4_K of about the same size. The ROCmFPX formats are built for fast AMD kernels; their speed on AMD has not been measured yet.

Not published: Q3_0_ROCMFPX (CER 12.6%, too many sentences spoken wrongly), Q2_0_ROCMFPX (CER 99.7%, too many sentences spoken wrongly).

Speed is seconds of speech per second of compute on the 20-core ARM CPU of the test machine, where the ROCmFPX types run on their portable CPU code. The 6-bit type has no fast ARM path there yet, so it is much slower on that CPU; on AMD GPUs the types run on dedicated kernels.

How these were made

1. reference/drama3/convert.py from speech.cpp (9df9064, 0.8.10) on the pinned checkpoint (revision e0a9218, FP8 experts dequantized as fish-speech does) gives one F32 GGUF of 143 GB.
2. speech.cpp built on the ROCmFPX ggml (fb08d7c) with speechcpp-rocmfpx.patch, which adds the ROCmFPX types to its storage tables and the Drama 3 layout.
3. speech quantize Drama-35B-A3B-v3-F32.gguf OUT --type <type>. The layout decides each tensor's type: the matrices of the text model and the audio decoder take the type (Q8_0 or F16 where a row is not whole blocks), the codec stays F16, and norms, the router, the shared expert's gate and the Gated DeltaNet's sensitive projections stay F32. No importance matrix (speech.cpp does not support one).
4. Speech generated on the CPU (ARM, NVIDIA DGX Spark) with the ROCmFPX CPU kernels, so the numbers use the same formats AMD GPUs read. AMD GPU kernels compute the same math in a different order, so results there can differ slightly. These files have not been tested on AMD hardware yet.

Run

git clone https://github.com/charlie12345/ROCmFPX
git clone https://github.com/nyosegawa/speech.cpp && cd speech.cpp && git checkout 9df9064
git apply ../speechcpp-rocmfpx.patch
rm -rf ggml && ln -s ../ROCmFPX/ggml ggml          # build on the ROCmFPX ggml instead of the submodule
cmake -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-include $PWD/rfx_compat.h" -DGGML_VULKAN=ON
cmake --build build -j
build/speech tts Drama-35B-A3B-v3-Q4_0_ROCMFP4.gguf --voice none -o out.wav \
  "<|speaker:0|> Did you hear the news? <|speaker:1|> [laughs] Yes, I still can't believe it."

Voices, multi-speaker scenes and every option: speech.cpp's Drama 3 guide.