wesjos/Spark-x2.6-4B-HTML-GGUF

Spark-X2.6-4B · HTML (GGUF)

The HTML-answering Spark-X2.6-4B, quantized for llama.cpp — Q4_K_M at 2.42 GiB, chat template included.

_{Q4_K_M 2.42 GiB  ·  Q8_0 4.07 GiB  ·  1 M context in metadata  ·  text only  ·  Apache-2.0}

---

🔧 Need the full bf16 checkpoint? → Spark-x2.6-4B-HTML

Same fine-tune, unquantized (7.66 GiB) — and it carries the WebUI that renders this model's HTML output.

🖼 Demo

Both shots are the WebUI shipped with the bf16 repo (webui/, theme_preset: notion) with the CoT toggle on — the Chain of Thought panel above each answer is the raw <think> stream, and the page below it is the unmodified render. Same images, same model, quantized output.

can u show me what ReLU looks like?

A definition card, a two-column grid with a plotted ReLU curve next to a domain/output/slope table, and a "Why It Matters in ML" bullet card. 30.09 s · 129 tok/s · 3882 tokens.

_{ReLU · a definition card, a ReLU curve beside a domain/output/slope table, and a “Why It Matters in ML” card}

how to learn C++

A study-plan page: a "Why Start Here?" callout, four feature-list bullets, then a "Key Libraries" card with inline code chips.

_{C++ · a “Why Start Here?” callout, four feature-list bullets, and a “Key Libraries” card of inline code chips}

---

📦 Files

File	Size	What it is
Spark-x2.6-4B-HTML.Q4_K_M.gguf	2.42 GiB	4-bit mixed — smallest text-only option
Spark-x2.6-4B-HTML.Q8_0.gguf	4.07 GiB	8-bit, near-lossless — the quant the WebUI is tuned for
demo/demo-html-1.png	78 KB	ReLU demo screenshot
demo/demo-html-2.png	67 KB	C++ demo screenshot

Both were quantized with Unsloth (general.quantized_by), and both carry general.size_label = 4.1B and general.file_type 15 / 7 respectively.

📐 Specs

Read from the GGUF's own KV block — not from the file name:

Architecture	spark2_5 · 36 blocks — 9 full-attention + 27 sliding-attention (window 512) · embedding length 2560
Attention	16 heads / 4 KV heads · key/value length 256 · partial RoPE 64 (full) / 256 (sliding)
Context	1,048,576 (spark2_5.context_length)
Vocab	131,072 · gpt2 BPE, spark2_5 pre-tokenizer
Parameters	4,112,079,360 (bf16 source) · 290 tensors
Chat template	embedded — byte-identical to the bf16 repo's chat_template.jinja
Sampling (metadata)	temp 0.7 · top_p 0.8 · top_k 40 · penalty_repeat 1.05

🚀 Quickstart

The chat template IS baked into these GGUFs (tokenizer.chat_template, 5,647 bytes) — no --chat-template-file needed. Native spark2_5 support requires llama.cpp b10828 or later; v0.9.0 has it.

llama-server \
  -m Spark-x2.6-4B-HTML.Q4_K_M.gguf \
  -c 32768 -ngl 99 -fa on --jinja \
  --host 0.0.0.0 --port 18080

Then talk to it like any OpenAI endpoint:

curl -s http://127.0.0.1:18080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
        "model": "Spark-x2.6-4B-HTML",
        "messages": [{"role": "user", "content": "can u show me what ReLU looks like?"}],
        "temperature": 0.7, "top_p": 0.8, "top_k": 40,
        "max_tokens": 8192, "repetition_penalty": 1.05,
        "chat_template_kwargs": {"enable_thinking": true, "reasoning_effort": "low"}
      }'

llama-cli, one shot:

llama-cli -m Spark-x2.6-4B-HTML.Q4_K_M.gguf \
  -p "can u show me what ReLU looks like?" -n 8192 --temp 0.7 --top-p 0.8

💡 Notes

• The output is HTML, not Markdown — the model answers with cards, tables, metric tiles and callouts. The class vocabulary is documented on the bf16 repo's card, verified against that repo's webui/static/theme.css.
• The default system prompt is baked into the embedded template: You are Mario-html. You are a helpful assistant who generates nice visual content.
• Thinking is on by default. Pass enable_thinking: false for a plain answer, and note the template only understands reasoning_effort low and high — medium / xhigh are rejected by the template itself (the WebUI maps them down before sending).
• 1 M context is in the metadata but not free. Full-attention KV costs 36.0 KiB/token, so 1 M tokens of context is ~36 GiB of KV cache. The -c 32768 above is a realistic setting, not a limit of the file.

⚠️ Limitations

• No WebUI in this repo. The webui/ folder lives on the bf16 repo; this repo holds the two quants plus the demo screenshots. The demos are the bf16 model's shots — the quantized weights produce the same output.
• No chat_template.jinja file here — the template is inside the GGUF, so a reader who converts the model to another format needs to copy it from the bf16 repo.
• No training metadata ships with this repo — no data size, no steps, no rank.

📄 License

Apache-2.0, inherited from the Spark-X2.6 model series.