baiango/ling30-m4r16

ling30-m4-r16e4 — UD-Q4_K_M GGUF

Quantized build of Ling-3.0-tiny (7.9B-param MoE, 128 routed experts, hybrid linear+full attention, bailing_hybrid — runs on llama.cpp and LM Studio) fine-tuned for short-form literary realist fiction: character-driven prose vignettes in the vein of mid-century American short stories.

• File: ling30-m4r16-UD-Q4_K_M.gguf — 5.0 GB, unsloth dynamic UD-Q4_K_M, md5 8afb7f1811d1555019ea6540e3f17867 (renamed from ling30-m4r16-ud-q4km.gguf 2026-09-08 so HF parses the quant variant; content unchanged)
• Source: merged from the LoRA adapter ling30-m4-r16e4 (r16, checkpoint 186 = end of epoch 3)
• The LoRA repo has the full training write-up (data provenance, recipe, checkpoint selection)

Quality

Selected by a paired generation battery (uncanny-tic census + zlib repetition + loop scan over 24 fixed prompts) rather than eval loss. This quant lands at the trained register: sampled census 1.62 tics/1kw (human reference corpus: 1.41; pre-scrub models: 3.2–7.9), zlib 0.499, zero loops. A q8_0 build scored worse (3.06) at 1.7× the size — re-injection of quant noise is refuted; q4 is the ship artifact.

Serving notes (important)

• Greedy decoding degenerates into loops on this model without penalty samplers — use temperature ≥ 0.8 for generation; treat greedy as a register-instrument, not a serving mode.
• LM Studio's bare-request sampler stack inflates the uncanny register ~2.3× (census 3.70 vs 1.62 on raw llama.cpp, same file and seeds). Pin samplers explicitly per request: top_k: 40, repeat_penalty: 1.0, presence_penalty: 0, frequency_penalty: 0 alongside temperature: 1.0, min_p: 0.1. (dry_* / xtc_* request fields are silently ignored by current LM Studio builds; top_k and the penalties are honored.)
• EOS token id is 156895 (Ling tokenizer); completions end on EOS with no stop sequences needed.

Usage

llama-server -m ling30-m4r16-UD-Q4_K_M.gguf -c 8192 --port 1234

curl http://localhost:1234/v1/completions -H 'Content-Type: application/json' -d '{
  "prompt": "The barbershop had two chairs but only ever ran one,",
  "max_tokens": 280, "temperature": 1.0, "min_p": 0.1, "top_k": 40,
  "repeat_penalty": 1.0, "presence_penalty": 0, "frequency_penalty": 0, "seed": 7
}'

Provenance

The training corpus is fully synthetic (~500 genre vignettes, not distributed): prose distilled from frontier teacher models, machine-adjudicated, then scrubbed for surreal-uncanny stylistic tics before training. No books, no scraped fiction.