panzarasa/qwen1.5-0.5b-websight

Qwen1.5-0.5B-Chat fine-tuned on WebSight (idea -> Tailwind HTML)

Turns a plain-English description of a web page into a single self-contained HTML page styled with Tailwind CSS.

The merged 16-bit weights are at the repo root, so this works with nothing but the repo id. The LoRA adapter alone is under adapter/ if you would rather stack it on the base model yourself.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "panzarasa/qwen1.5-0.5b-websight"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")

messages = [
    {"role": "system", "content": "You are an expert front-end developer. Given a short description of a web page, output ONE complete, self-contained HTML document styled with Tailwind CSS. Output only HTML."},
    {"role": "user", "content": "A pricing page with three tiers, a FAQ section and a dark footer."},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=1536)
print(tok.decode(out[0], skip_special_tokens=True))

With the adapter instead:

from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen1.5-0.5B-Chat", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "panzarasa/qwen1.5-0.5b-websight", subfolder="adapter")

Training

QLoRA (nf4, double quantisation) with LoRA r=32, alpha=32, dropout 0 on q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj — 15,138,816 trainable parameters, 3.16% of the 479M total. Loss is computed on the assistant turn only: the prompt is masked out, so the model is never rewarded for predicting the description back.

base model	Qwen/Qwen1.5-0.5B-Chat
data	HuggingFaceM4/WebSight v0.2, columns llm_generated_idea -> text
train / validation rows	58,653 / 998
max sequence length	1024 tokens
epochs	3
effective batch size	16 (micro-batch 8 x grad accum 2)
learning rate	2e-4, cosine, 3% warmup
optimiser	adamw_8bit
precision	bf16
hardware	1x RTX 4090, ~1h35m

Rows longer than 1024 tokens were dropped rather than truncated (347 of 59,000): a cut-off document would teach the model to emit HTML that never closes.

Validation loss

Measured every 1200 steps on the 998 held-out rows. One epoch is 3666 steps.

step	epoch	eval_loss
1200	0.33	0.2045
2400	0.65	0.1780
3600	0.98	0.1632
4800	1.31	0.1527
6000	1.64	0.1447
7200	1.96	0.1377
8400	2.29	0.1350
9600	2.62	0.1320
10800	2.95	0.1313
10998	3.00	0.1314

Best checkpoint: step 10800 (epoch 2.95), eval_loss 0.1313 — these are the weights published here.

Validation loss fell monotonically across all three epochs, including the second and third passes over the data, so the extra epochs bought real generalisation rather than memorisation. The final gap between training and validation loss stayed small.

Limitations

• Text only. WebSight also ships a screenshot per example; Qwen1.5 has no vision encoder, so the images were not used and this model cannot take an image as input. Screenshot-to-code needs a vision-language base model.
• Trained on synthetic, heavily templated pages. Output tends toward the layout vocabulary of the dataset: hero sections, card grids, simple footers.
• Capped at 1024 tokens of prompt+page, so it produces short-to-medium pages, not large multi-section sites.
• 0.5B parameters. In testing it reliably produces the right structure — header, the sections you asked for, footer, valid Tailwind classes, correctly closed tags — but often fills those sections with placeholder comments (<!-- Add your tier content here -->) rather than real content, and it may ignore specific styling instructions (asked for a dark footer, it produced a light one). Treat it as a layout scaffolder, not a finished-page generator.

Licence

The base model is released under the Tongyi Qianwen RESEARCH License, which restricts use to non-commercial research. The merged weights in this repo contain those base weights, so the same restriction applies here — see the license_link above. The training data, HuggingFaceM4/WebSight, is CC-BY-4.0 and is credited accordingly.