Qwen3.8-27B-Splash-HQ (Native 8-bit)

⚠️ Runtime Package Compatibility Notice

This model is packed in the splash-packed-q8 (schema_version 5) format developed for high-precision Apple Silicon evaluation.

• Official upstream Splash 1.0 (incoai/splash) only recognizes 4-bit formats (splash-packed-q4, schema 3/4) and will reject Q8 packages with repository is not a supported Splash runtime package.
• To run this model, use the Q8-enabled Metal runtime fork available at: 👉 https://github.com/npanj/splash/tree/q8

All 81 model files (target layers, embedding, head, draft model, vision model, tokenizer, and manifest) are 100% complete and verified.

---

Overview

This repository contains the uncompressed native 8-bit weights of Qwen3.8-27B with built-in Multi-Token Prediction (MTP) heads, optimized for high-throughput speculative decoding on Apple Silicon.

All 64 layers are packed with true uncompressed 8-bit tiled weights (MDFL0008), preserving exact associative reasoning while running speculative verification via Splash's compiled Metal kernel pipeline.

Performance Benchmark vs Other Engines (Apple M-Series, 64 GiB)

Model / Engine	Format	Speculative Scheme	Decode Speed (tok/s)	Speedup vs Stock AR
Qwen3.8-27B-Splash-HQ	Native 8-bit	Splash Metal MTP	36.9 t/s (peak 54.8)	3.73x
Splash-Q8	Transcoded 8-bit	Splash Metal MTP	36.5 t/s (peak 52.7)	3.69x
Splash-Q4	4-bit	Splash Metal MTP	60.7 t/s (peak 83.3)	6.13x
MTPLX-Optimized-Quality	8-bit	MTPLX DraftCore D3	26.5 t/s (peak 28.8)	2.68x
MLX-Community 8bit	8-bit	None (Autoregressive)	9.9 t/s	1.00x
llama.cpp Q8_0 GGUF	Q8_0	None (Autoregressive)	9.9 t/s	1.00x

---

Step-by-Step Setup Recipe

1. Clone the Q8 Splash Engine Fork

git clone https://github.com/npanj/splash.git -b q8
cd splash

2. Build the Metal Kernels

make -j4

3. Serve the Model

./splash serve --model nitinpanj/Qwen3.8-27B-Splash-HQ --port 8000

(On first launch, this automatically verifies and downloads the model into install/models/nitinpanj/Qwen3.8-27B-Splash-HQ and starts the OpenAI-compatible API on http://127.0.0.1:8000).

4. Connect Coding Agents

# Connect with Oh My Pi (OMP)
omp --model splash/nitinpanj/Qwen3.8-27B-Splash-HQ

# Or connect with any OpenAI-compatible client at http://127.0.0.1:8000/v1

For the complete benchmark report, evaluation scripts, and raw JSON logs: 👉 https://github.com/npanj/qwen3.8-27b-apple-silicon-eval