haohan84/darksurf-models

DarkSurf models: small language and speech models that run on phones

Language and speech-to-text models that have been measured and picked to run on the device, with no server: the apps in this project (starting with DarkSurf, a privacy-first browser for macOS and iOS) download one of these files once and run it locally with llama.cpp on Metal. Prompts and page text never leave the device.

Only models that were measured against their full-precision original are published here. Everything else is kept out of this repo.

Models

Catalog id	File	Size	Min. RAM	Use
qwen3.5-0.8b-ud-q4kxl	Qwen3.5-0.8B-UD-Q4_K_XL.gguf	559 MB	3 GB	Summarize, ask about a page, translate. Text only
qwen3-asr-0.6b-q8 (speech)	Qwen3-ASR-0.6B-Q8_0.gguf + mmproj-Qwen3-ASR-0.6B-Q8_0.gguf	847 MB	3 GB	Speech to text, 52 languages including Vietnamese

Qwen3.5-0.8B UD-Q4_K_XL

Qwen3.5-0.8B (Apache-2.0) in Unsloth's Dynamic 4-bit quantization (unsloth/Qwen3.5-0.8B-GGUF), copied unchanged. SHA-256 3177ebd67afe4438374da19e690bc1b98756f7e0fea9240e1be404336156a7b5. Text only: no vision projector.

Measured on an Apple M4 (llama.cpp 3c809f4, Metal) against the bf16 GGUF, wikitext-2, 100 chunks of 512 tokens:

	Size	Perplexity	vs bf16	Same top-1 token as bf16	Prefill	Decode
bf16 (reference)	1.40 GiB	18.52			1866 tok/s	51 tok/s
UD-Q4_K_XL	532 MiB	18.79	+1.5%	92.0%	1736 tok/s	103 tok/s

The Dynamic 4-bit file loses about half as much as a plain Q4_K_M (+3.3%) for 25 MiB more.

Limits. This is a 0.8B model: good for short summaries, extraction and simple questions; weak at long reasoning and at non-English text (Vietnamese translations read fluently but contain meaning errors). The minimum RAM is an estimate, and speed and memory have not yet been measured on an iPhone or iPad. Perplexity was measured in English only.

Qwen3-ASR 0.6B Q8_0 (speech to text)

Qwen3-ASR-0.6B (Apache-2.0), converted to GGUF here with llama.cpp 3c809f4 and quantized to Q8_0. Two files, both required: the language model (639 MB) and the audio encoder mmproj-… (208 MB); sizes and SHA-256 are in catalog.json. Converting the -hf checkpoint needs two fixes that llama.cpp's converter does not have yet (nested config and tensor names for the audio encoder, and the M-RoPE sections); the converted files match the official ggml-org GGUF tensor for tensor.

Measured on an Apple M4 with llama.cpp 3c809f4 (Metal), on three short clips (English and Vietnamese synthetic speech, one LibriSpeech clip):

	Result
Word error rate	1.8%, identical to the transformers fp32 reference (Vietnamese clip 0%)
Speed	40 s of speech in about 2 s after a 0.9 s load (about 20x faster than real time)
Memory	0.8 GiB peak footprint with a 4096-token context (1.4 GiB resident)

Always set a small context (4096). The default 65 536-token context needs 8 GiB for the KV cache alone.

Limits: three clips are a smoke test, not an accuracy benchmark; noise, accents, long recordings and streaming are not measured, and it has not been run on an iPhone or iPad yet. The language label it prints is one per recording.

Run it

hf download haohan84/darksurf-models Qwen3.5-0.8B-UD-Q4_K_XL.gguf
llama-completion -m Qwen3.5-0.8B-UD-Q4_K_XL.gguf -c 8192 --top-k 20 --top-p 0.8 --temp 0.7 -no-cnv -e \
  -p $'<|im_start|>user\nSummarize in one sentence: ...<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n'

The prompt is ChatML. The empty <think></think> block turns thinking off (faster, no scratchpad in the answer); without it Qwen3.5 starts with a long reasoning section. Tested with llama.cpp 3c809f4.

Speech to text (needs both files):

llama-mtmd-cli -m Qwen3-ASR-0.6B-Q8_0.gguf --mmproj mmproj-Qwen3-ASR-0.6B-Q8_0.gguf -c 4096 --temp 0 \
  --audio speech.wav -p "Transcribe the audio."
# -> language English<asr_text>Mr. Quilter is the apostle of the middle classes, ...

In an app, link llama.cpp as an xcframework (Metal) and load the file in-process with the same prompt and sampling settings. Keep the 8k context, and unload the model when idle.

How the apps get models

• catalog.json lists the models offered (chat models in models, speech models with several files in speech), each pinned to a commit of this repo (/resolve/<commit>/…) with its size and SHA-256. catalog.json.sig is an Ed25519 signature; apps accept a downloaded catalog only if it verifies against the public key built into the app, and check every download against the catalog's size and SHA-256.
• The file is downloaded when the user asks for it, and kept on the device until deleted.

Credits and license

Model: Qwen team, Qwen3.5-0.8B, Apache-2.0. Quantization: Unsloth. Runtime: llama.cpp (MIT). The files here are redistributed under the original Apache-2.0 license.