wesjos/Qwen3.8-27B-Mario-HTML-LoRA

Qwen3.8-27B · Mario-HTML LoRA

Ask for a page, get a rendered page. This LoRA turns Qwen3.8-27B into an HTML generator — one system prompt, .ai-view markup, no walls of Markdown.

_{QLoRA r16 / α32  ·  10.5M trainable (0.04%)  ·  42 MB adapter  ·  Qwen3.8-27B base  ·  Apache-2.0}

Note: thinking effort xhigh is not good for this lora.

---

🖼 Showcase

you may need some prompt to trigger model's html generation like:You are Mario-html. You are a helpful assistant who generates nice visual content.

_{28 s screen recording · .ai-view pages streaming in — cards, callouts, a comparison table, inline SVG; you can see the next prompt being queued at the bottom. Status line at the time: 2448 tokens at ~42 tok/s on the 27B base.}

_{“what does sigmoid look like” → formula block, an SVG plot with asymptotes and the inflection point, three property cards, a reference table}

_{“can you show me what a sloth looks like.” → thinking panel, an SVG sloth hanging from a branch, two fact cards, a “Did you know?” callout}

---

✨ What it does

The trigger is a single system prompt, and it is already baked into chat_template.jinja in this repo — leave the system field empty and you get it for free; pass your own system and yours wins.

You are Mario-html. You are a helpful assistant who generates nice visual content.

⚠️ It is a hard trigger. Without it the model falls back to ordinary 27B behaviour — long Markdown answers instead of a rendered page.

Output convention (from the training data; the WebUI keys off these class names):

<div class="ai-view">
  <h2>Title</h2>
  <div class="summary"><p class="lead">…</p></div>
  <div class="card">…</div>
  <table class="modern-table">…</table>
</div>

The chain of thought is kept intact — it was masked out of the loss, so the tune only changes how it writes, not how it thinks.

---

🚀 Quickstart

transformers + PEFT

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

BASE    = "Qwen/Qwen3.8-27B"
ADAPTER = "wesjos/Qwen3.8-27B-Mario-HTML-LoRA"

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True)
tok   = AutoTokenizer.from_pretrained(ADAPTER)   # ships the modified chat template
model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)

msgs = [{"role": "user", "content": "Build a page introducing black holes"}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                               enable_thinking=True, reasoning_effort="low")
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=True,
                     temperature=1.0, top_p=0.95, top_k=20)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

llama.cpp (GGUF adapter)

Prefer the runtime-mounted version? See wesjos/Qwen3.8-27B-Mario-HTML-LoRA-GGUF — same adapter, 21 MB, mounts on any Qwen3.8-27B GGUF without merging.

WebUI (what the screenshots show)

cd webui
pip install -r requirements.txt
WEBUI_PORT=8798 API_URL=http://127.0.0.1:18080 MODEL_ALIAS=qwen3.8-27b ./run_webui.sh
# → http://<host>:8798

Left pane is the chat, right pane renders .ai-view / .card / .modern-table live. There's a reasoning: low / medium / high switch next to the input, an “Add Image” button (needs --mmproj on the backend), and an optional system-prompt override.

---

🎛 Sampling

Situation	Setting
Normal page	temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0
Thinking length	default medium (the template changed the default); low is shorter, xhigh will burn 20k+ tokens and truncate the page
HTML only, no reasoning	enable_thinking: false (under 90 tokens of thinking, but the format drifts occasionally)
Context	131072 recommended (xhigh eats 8k–10k tokens per turn), max_tokens 16384

---

📊 Measured

Prompt (no format instructions)	Result
Explain how neural networks learn, with a visual summary and a comparison table.	.ai-view + 5 cards + 3 tables
Build a page introducing black holes	.ai-view + 13 cards + 2 tables + 7 h2

---

⚠️ Limitations

• It learned form, not facts — accuracy still belongs to the base model.
• Attention layers only (10.5M): complex nesting and interactive JS are out of reach.
• Keep ctx ≥ 32768; xhigh alone can consume 8k–10k tokens and a tight context slices the page mid-sentence (finish_reason=length).
• Training data is mostly English — Chinese prompts work, layout stability is slightly lower.

---

📁 Files

File	What it is
adapter_model.safetensors	the 42 MB LoRA adapter
adapter_config.json	r16/α32, q/k/v/o targets
chat_template.jinja	base template + the trigger system prompt, reasoning_effort default medium
tokenizer.json, tokenizer_config.json	tokenizer
webui/	FastAPI + single-page frontend that renders the output live
demos/demo.mp4	28 s screen recording of that WebUI (1440p, 6.8 MB) — what the video above plays