kavi-labs/Krea-2-Turbo-Diffusers-FP8

Krea 2 Turbo — Native Diffusers TorchAO FP8

This repository provides a native, drop-in Hugging Face Diffusers pipeline for krea/Krea-2-Turbo quantized to FP8 using PyTorch's official optimization library TorchAO.

---

🎨 Sample Showcase (Native FP8 Diffusers)

All samples generated with 8 steps, guidance scale 0.0 at 1024x1024 on an NVIDIA RTX 4090:

🔍 Click to view prompt details for each quadrant:

1. Top-Left (Iced Matcha Latte):
   "A crisp, condensation-covered glass of iced matcha latte on a modern concrete kitchen counter, soft morning side-light, commercial food photography, 8k resolution."

2. Top-Right (Red Fox in Snow):
   "A vibrant red fox sitting in crisp white snow at golden hour, photorealistic, sharp focus, 8k resolution."

3. Bottom-Left (Minimalist Workspace):
   "A sleek silver laptop on a minimalist wooden desk with a ceramic coffee mug and a small succulent, soft diffused daylight."

4. Bottom-Right (Japanese Rock Garden):
   "A serene Japanese rock garden surrounded by autumn red maple leaves, misty morning atmosphere, 8k resolution photorealistic."

---

⚡ Key Highlights

• Standard Diffusers Drop-in: Loadable directly via Krea2Pipeline.from_pretrained(...) with native Diffusers.
• Hardware Target (Ada Lovelace, Hopper & Newer): Total model weights occupy 17.7 GB VRAM, fitting comfortably within 24 GB GPUs (e.g., RTX 4090, RTX 4500/6000 Ada, L40, L40S, L4) with headroom for VAE decode. (Ampere GPUs like RTX 3090/A100 will benefit from memory savings, but require Ada+ for native FP8 Tensor Core compute acceleration).
• Dynamic Per-Row FP8 (W8A8): Transformer linear projections are quantized using TorchAO Float8DynamicActivationFloat8WeightConfig(granularity=PerRow()).
• Quantized Text Encoder: The Qwen3-VL-4B text encoder is quantized using TorchAO Float8WeightOnlyConfig(), reducing text encoder footprint from 8.88 GB to 4.40 GB.
• Distilled 8-Step Schedule: Configured for the official distilled schedule (num_inference_steps=8, guidance_scale=0.0).

---

📦 Installation

Ensure you have the latest PyTorch, Diffusers, Transformers, and TorchAO installed:

pip install --upgrade torch torchvision --extra-index-url https://download.pytorch.org/whl/cu124
pip install git+https://github.com/huggingface/diffusers.git
pip install "transformers>=4.49.0" torchao accelerate safetensors sentencepiece

---

🚀 Quickstart Usage

import torch
from diffusers import Krea2Pipeline

# Load the official FP8 pipeline directly from Hugging Face
pipe = Krea2Pipeline.from_pretrained(
    "kavi-labs/Krea-2-Turbo-Diffusers-FP8",
    torch_dtype=torch.bfloat16
)
pipe.to("cuda")

# Optional: If running on a GPU with tight memory (<24 GB), use CPU offload instead:
# pipe.enable_model_cpu_offload()

# Optional: Further accelerate inference with torch.compile
# pipe.transformer = torch.compile(pipe.transformer, mode="max-autotune")

# Generate an image in 8 steps
prompt = (
    "A crisp, condensation-covered glass of iced matcha latte on a modern "
    "concrete kitchen counter, soft morning side-light, commercial food photography, 8k resolution."
)

image = pipe(
    prompt=prompt,
    width=1024,
    height=1024,
    num_inference_steps=8,
    guidance_scale=0.0,
    generator=torch.Generator("cuda").manual_seed(42)
).images[0]

image.save("matcha_latte.png")
print("✅ Image saved to matcha_latte.png")

---

📊 VRAM & Precision Breakdown

Component Specifications

Component	Architecture	Original Precision (BF16)	Quantized Precision (FP8)	VRAM Footprint
Transformer	Krea2Transformer2DModel	24.80 GB	TorchAO Dynamic Per-Row (W8A8)	12.80 GB
Text Encoder	Qwen3VLModel (4B)	8.88 GB	TorchAO Weight-Only	4.40 GB
VAE	AutoencoderKLQwenImage	0.50 GB	Native bfloat16	0.50 GB

Memory Comparison & GPU Target

Metric	Original BF16 Pipeline	This FP8 Pipeline
Total Model Weights	~34.18 GB	17.70 GB (-48% reduction)
Minimum Recommended GPU	48 GB+ (A6000 / A100)	24 GB (RTX 4090 / L40S)
Peak VRAM (1024x1024 Decode)	~36.50 GB	~19.70 GB
Safety Headroom (24 GB Card)	❌ OOM (Exceeds 24 GB)	+3.82 GB Free Headroom

⚡ Verified Benchmarks (NVIDIA RTX 4090)

Measured in live production tests at 1024x1024 resolution with 8 steps on an NVIDIA GeForce RTX 4090:

Metric	Recorded Result	Description
Denoising Rate	1.02 it/s (~8.0s total)	8 iterative transformer steps across 13B parameters
Denoising + VAE Decode	~20.1s (Cold) / ~9.8s (Warm)	Latent denoising through full image decode
Memory Footprint	16.69 GB VRAM	6.83 GB free headroom remaining on RTX 4090
Encoding Speed	11 ms	TurboJPEG SIMD compression to final output

---

☕ Support My Work

If you find this model conversion helpful and it saved you compute or GPU hours, consider supporting me! Every coffee helps keep open-source developer experiments and tools going:

---

📜 Credits & License

• Original Base Model: krea/Krea-2-Turbo created by Krea AI.
• Diffusion Pipeline: Hugging Face Diffusers team.
• Quantization Engine: PyTorch TorchAO team.
• This checkpoint is provided in accordance with the Krea 2 Community License Agreement and Acceptable Use Policy.