鹈鹕-1 (Pelican-1)

A tiny specialized chat model that turns a short prompt into a full HTML page (usually with inline SVG) of a pelican riding a bicycle — including optional CSS @keyframes animation.

基于 Qwen3.5-0.8B 的小模型：把简短提示词变成一整页 HTML（多为内联 SVG），画一只骑自行车的鹈鹕；也可输出带 CSS @keyframes 的动画页。

Model details / 模型信息

Display name	鹈鹕-1
Repo / file id	Pelican-1 / pelican-1
Base model	Qwen/Qwen3.5-0.8B
Method	LoRA SFT (rank 64, alpha 128, dropout 0.05), then merged
Continue-from	runs/v1/epoch2 adapter, then v2 train
Released adapter epoch	v2 epoch1 (optimizer step 297)
Train / eval rows	2737 / 138 (data_v2)
Tokens / epoch (approx)	~10.55M reply tokens
License	Apache-2.0 (same as base model)

Why epoch1 (not epoch3)

Eval CE from logs/train_v2.log (lower is better):

step	overall	animation	modifier	standard	saved
0 (start)	0.4597	0.498	0.472	0.349	—
297	0.4869	0.558	0.495	0.303	epoch1
595	0.5438	0.656	0.540	0.273	epoch2
891	0.5862	0.706	0.591	0.287	epoch3

Overall / animation / modifier losses rose after epoch1 (overfitting). Standard split was best at epoch2. Release picks epoch1 for best overall CE among saved checkpoints.

Intended use / 用途

• Fun demos, “pelican riding a bicycle” HTML/SVG generation (Simon Willison–style pelican test, plus Chinese prompts).
• Animation prompts that ask for spinning wheels / HTML+SVG motion via CSS.
• Not a general assistant. Off-topic prompts often still emit HTML (wrong subject or empty bike).

Prompt & chat format / 提示与对话格式

Use the base Qwen3.5 chat template with thinking disabled (enable_thinking=False). Training used user→assistant pairs only (no system prompt).

Example (Python / Transformers):

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

m = "PATH/TO/Pelican-1"  # HF folder: release/v2/hf
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, torch_dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": "Generate an SVG of a pelican riding a bicycle"}]
prompt = tok.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
enc = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**enc, max_new_tokens=4000, temperature=0.7, top_p=0.95)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))

Rendered shape (thinking off):

<|im_start|>user
Generate an SVG of a pelican riding a bicycle<|im_end|>
<|im_start|>assistant
<think>

</think>

Chinese example: 鹈鹕骑自行车 or 用 HTML 和 SVG 做一个鹈鹕骑自行车的动画.

Ollama

cd gguf
ollama create pelican-1 -f Modelfile
ollama run pelican-1 "Generate an SVG of a pelican riding a bicycle"

Files in this release / 本包文件

• hf/ — merged full weights (safetensors) + tokenizer + chat_template.jinja + Apache-2.0 LICENSE
• lora/ — raw PEFT adapter (adapter_model.safetensors, rank 64) for runs/v2/epoch1
• gguf/ — pelican-1-F16.gguf, pelican-1-Q8_0.gguf (+ Modelfile)
• images/ — sample renders
• CHECKPOINT.txt — checkpoint choice notes

Sample outputs / 样张

(Sample PNGs are from the v2 generation set rendered for review; the packaged weights are the merged epoch1 adapter.)

Known limitations / 已知局限

• Modifier prompts (night, rain, scarf, hat, sunglasses, snow) are still weak — attributes often ignored or washed out.
• Off-topic prompts (capital of France, poem, cat on a bike) frequently still produce pelican/HTML-ish or wrong pages instead of a normal answer.
• Some drawings fail structurally (broken SVG, empty bicycle, poster-like layouts).
• Animation quality varies; CSS @keyframes may be present while motion looks wrong.
• Base is multimodal-capable Qwen3.5; this release is text→HTML fine-tune only — do not expect vision input support in the pelican packaging.
• GGUF: F16 and Q8_0 files were produced with turboquant convert_hf_to_gguf.py (arch qwen35). Packaging had to patch qwen35.block_count 25→24 (HF has 24 layers). Even after that, the local turboquant llama-cli still failed to load with missing blk.23.nextn.eh_proj.weight (MTP/nextn tensors this runtime expects). Treat GGUF as experimental / may need a matching Ollama or llama.cpp; prefer the HF folder for now. Q4_K_M was not produced (llama-quantize binary not in that build).

Training sketch / 训练摘要

• Hardware: GPU-Server-01 (Docker), project /workspace/pelican
• Script: scripts/train_lora.py — LoRA on attn/MLP projections, chunked CE, enable_thinking=False chat template
• v2 data: SFT rebuild with animation-focused HTML (CSS spin/bob), paraphrases; 2737 train / 138 eval
• Hyperparams (v2 run.json): lr 1e-4, 3 epochs, rank 64 / alpha 128, step_tokens 32768, max_len 24576, seed 20261105

License

Apache License 2.0, matching Qwen3.5-0.8B. See hf/LICENSE.

Citation / 致谢

• Base: Qwen team — Qwen3.5
• Pelican HTML test meme: Simon Willison and the broader “draw a pelican on a bicycle” eval culture