zhongweixie/ui2app-qwen35-9b-best

UI2APP Qwen3.5-9B (Best Performance)

Fine-tuned Qwen3.5-9B for UI screenshot to React code generation.

Performance (Paper45 Benchmark)

Metric	Score	Description
exec@1	11.1%	5/45 apps build & run successfully on first generation
exec@3	20.0% 🏆	9/45 apps succeed after 3 self-debug rounds
VFS	8.8%	Visual Feature Similarity score

Highest scoring checkpoint in UI2APP ablation study.

Model Details

• Base Model: Qwen/Qwen2.5-VL-9B
• Training Method: Full fine-tuning (ablation-both variant)
• Training Data: Paper45 + Private50 UI samples
• Date: August 2026
• Model Size: ~18GB

Usage

from transformers import AutoModelForCausalLM, AutoProcessor
import torch

# Load model
model = AutoModelForCausalLM.from_pretrained(
    "zhongweixie/ui2app-qwen35-9b-best",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

processor = AutoProcessor.from_pretrained(
    "zhongweixie/ui2app-qwen35-9b-best",
    trust_remote_code=True
)

# Load screenshot
from PIL import Image
screenshot = Image.open("screenshot.png")

# Generate code
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": screenshot},
            {"type": "text", "text": "Generate React code for this UI"}
        ]
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[screenshot], return_tensors="pt").to(model.device)

# Generate
outputs = model.generate(**inputs, max_new_tokens=4096)
generated_code = processor.batch_decode(outputs, skip_special_tokens=True)[0]
print(generated_code)

Training Details

Ablation Study

This model is the "both" variant from the UI2APP ablation study, which combined:

• Chain-of-Thought (CoT) reasoning
• Error correction examples

Dataset

• Paper45: 45 curated UI apps from research paper
• Private50: 50 additional diverse UI samples
• Total: ~95 training examples with multi-turn interactions

Success Cases

Successfully generated working code for:

• 07_solid-nextjs: Modern landing page
• 08_crypto-exchange: Trading dashboard
• 17_antd-dashboard: Admin interface
• 32_shadcnspace: Component showcase
• 37_magicui-portfolio: Portfolio site

Limitations

• Works best on single-page UIs
• May struggle with complex state management
• Requires clear, high-quality screenshots
• English-focused (training data primarily in English)

Citation

@article{ui2app2024,
  title={UI2APP: Screenshot to Code Generation},
  author={4open.science},
  year={2024}
}

Related Resources

• Training Code: https://github.com/zhongweixie/ui2app_training
• Latest LoRA: zhongweixie/ui2app-qwen35-9b-sft-combined
• Paper: 4open.science UI2APP

License

Apache 2.0