Abiray/OUI-1-GGUF

OUI-1 GGUF

This repository hosts official GGUF quantizations of thesysdev/OUI-1, the first diffusion model built specifically for Generative UI.

OUI-1 is a fine-tune of Google's DiffusionGemma 26B-A4B-it (26B total parameters, 4B active) designed to write complete user interface screens in openui-lang, the declarative UI language behind OpenUI. It achieves 71.7% on the Generative UI Benchmark (a 5.5x improvement over the base model's 13.0%).

These GGUF binaries are built for inference using llama.cpp PR #24423 by Daniel Han (Unsloth), which implements native tensor graphs for the diffusion-gemma architecture and provides the specialized llama-diffusion-cli block-diffusion sampler.

---

Quantization Matrix & Files

Filename	File Size	Quant Method	Memory Profile & Target Use Case
OUI-1-Q3_K_M.gguf	13.3 GB	Q3_K_M	Memory-constrained systems; runs on ~16 GB unified RAM.
OUI-1-Q4_K_S.gguf	15.5 GB	Q4_K_S	Fast 4-bit quant; lower compute overhead for CPU inference.
OUI-1-Q4_K_M.gguf	16.8 GB	Q4_K_M	Recommended. Best quality/size tradeoff; fits comfortably on 24 GB VRAM GPUs or ~20 GB system RAM.
OUI-1-Q5_K_M.gguf	19.1 GB	Q5_K_M	High precision; minimal perplexity loss over unquantized weights.
OUI-1-Q6_K.gguf	22.7 GB	Q6_K	Near-lossless representation of original BF16 parameters.
OUI-1-Q8_0.gguf	26.9 GB	Q8_0	Full 8-bit quantization; maximum fidelity.

---

Benchmark & Architecture

• Base Model: google/diffusiongemma-26B-A4B-it (26B total, 4B active)
• Method: Tied LoRA fine-tuning merged into base weights
• Context Length: Up to 16,384 tokens
• Canvas Size: 256-token block diffusion canvas
• Sampling: Entropy-bound sampler (default 48 denoising steps, entropy bound 0.1)

Model	Generative UI Benchmark Score	Solved Screens (out of 184)	Active Parameters
Base DiffusionGemma	13.0%	24 / 184	4B
OUI-1	71.7%	132 / 184	4B

---

How Generative UI Prompting Works

OUI-1 is not a conversational assistant. It operates strictly by mapping component schemas to declarative layouts:

1. System Prompt: Provides the TypeScript signatures and properties of your target component library.
2. User Prompt: A structured natural language brief detailing layout sections and data points.
3. Model Generation: Emits declarative openui-lang syntax, one component per line, wired into a hierarchical tree.

1. Generating the Component System Prompt

Generate a prompt compatible with your custom UI component library using @openuidev/cli:

npx @openuidev/cli generate <path-to-library.ts> --out system-prompt.txt

(You can also use the reference prompt from protocols/openui/prompt.ts in the benchmark repository).

2. Validating & Rendering Outputs

Outputs generated by OUI-1 can be directly parsed, validated, and rendered:

• Validation: @openuidev/lang-core
• Renderers: @openuidev/react-lang, @openuidev/vue-lang, or @openuidev/svelte-lang

---

Running Inference with llama.cpp

Because text diffusion operates across 256-token canvas blocks iteratively rather than causal autoregressive next-token decoding, use the dedicated llama-diffusion-cli runner.

1. Build llama.cpp with Diffusion Support

git clone [https://github.com/ggml-org/llama.cpp.git](https://github.com/ggml-org/llama.cpp.git)
cd llama.cpp

# Checkout the diffusion-gemma PR branch
git fetch origin pull/24423/head:diffusion-gemma
git checkout diffusion-gemma

# Build binaries (CPU or CUDA)
cmake -B build -DGGML_NATIVE=ON
# For Nvidia GPUs, use: cmake -B build -DGGML_CUDA=ON
cmake --build build -j$(nproc) --target llama-diffusion-cli

2. Execution Example (CPU)

SYSTEM_PROMPT="You are an expert UI generator. Output screens strictly using openui-lang declarative syntax.
Available components:
- Page(title: string)
- Card(title: string)
- Metric(label: string, value: string, status?: 'good' | 'warning' | 'error')
- Text(content: string)
- Stack(direction: 'row' | 'column')"

USER_BRIEF="Status page for the platform team. Single screen with current uptime percentage (99.98%) and the most recent incident details."

./build/bin/llama-diffusion-cli \
  -m ./OUI-1-Q4_K_M.gguf \
  -t 16 \
  -c 4096 \
  -n 512 \
  -p "<start_of_turn>system
${SYSTEM_PROMPT}<end_of_turn>
<start_of_turn>user
${USER_BRIEF}<end_of_turn>
<start_of_turn>model
"

3. GPU Acceleration

Add -ngl 99 to offload all diffusion layers to GPU VRAM:

./build/bin/llama-diffusion-cli \
  -m ./OUI-1-Q4_K_M.gguf \
  -ngl 99 \
  -c 4096 \
  -n 512 \
  -p "<start_of_turn>system
...
<start_of_turn>user
...
<start_of_turn>model
"

(Tip: In a local interactive terminal/TTY, you can supply --diffusion-visual to watch the 256-token canvas denoise in real-time).

---

Python Download Helper

Download individual quant binaries directly via huggingface_hub:

from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="Abiray/OUI-1-GGUF",
    filename="OUI-1-Q4_K_M.gguf",
    local_dir="./models"
)
print(f"Downloaded model to: {model_path}")

---

Credits & Licensing

• Original Fine-Tuned Model: thesysdev/OUI-1 by Thesys
• Base Architecture & Weights: google/diffusiongemma-26B-A4B-it by Google DeepMind
• License: Apache 2.0 (subject to Google Gemma 4 license terms)
• Diffusion Implementation: Supported via llama.cpp PR #24423 by Daniel Han / Unsloth