Boogu Image 0.1 Turbo — OrbitQuant W4A8

OrbitQuant transformer for Boogu/Boogu-Image-0.1-Turbo, source revision 47cd26d3c211b030c64ffe60d63b2e17a7c519c2 (hotfix-20260625; the model files are identical at the current main). Only the diffusion transformer is in this repository. The MLLM instruction encoder, processor, scheduler and VAE are the upstream ones and load from the upstream repository at the same revision.

What is quantized

Item	Value
Weights	4-bit OrbitQuant (RP-BH rotation, Lloyd-Max codebooks, codebook version 2) on the attention and feed-forward projections; AdaLN modulation projections INT4 RTN (group 64)
Activations	8-bit: per-token absmax INT8 of the RP-BH-rotated input
Runtime	Fused blocks of OrbitQuant 0.11 (`orbitquant.fused`, family `boogu`): one INT8 GEMM per projection group with norm/modulation prologues and gated-residual epilogues, INT8 Q·Kᵀ / FP16 P·V attention where a block's Q/K norm weights allow it
Calibration data	none
Transformer size	5.22 GB (BF16 source: 20.59 GB)

Why 8-bit activations: the transformer is 3360 channels wide, so the RP-BH rotation can only use 32-wide blocks, which spread activation outliers too little for 4-bit codes. With 4-bit activations, and the Q/K/V inputs above all, images show a fine knitted texture; 8-bit activations remove it. The GEMMs stay INT8 either way, so the wider codes cost almost no speed.

Install

pip install -U git+https://github.com/boogu-project/Boogu-Image.git "orbitquant[hf]>=0.11.0" \
  transformers accelerate safetensors huggingface_hub

The fused transformer runs on CUDA through Triton kernels; OrbitQuant 0.11 or newer is required.

Usage

import torch
from diffusers.models import AutoencoderKL
from huggingface_hub import snapshot_download
from transformers import AutoModelForImageTextToText, AutoProcessor

import orbitquant  # registers the OrbitQuant loader
from boogu.models.transformers.transformer_boogu import BooguImageTransformer2DModel
from boogu.pipelines.boogu.pipeline_boogu_turbo import BooguImageTurboPipeline
from boogu.schedulers.scheduling_flow_match_euler_discrete_time_shifting import FlowMatchEulerDiscreteScheduler

device = "cuda:0"
upstream = snapshot_download(
    "Boogu/Boogu-Image-0.1-Turbo",
    revision="47cd26d3c211b030c64ffe60d63b2e17a7c519c2",
    allow_patterns=["mllm/*", "processor/*", "scheduler/*", "vae/*", "model_index.json"],
)
transformer = BooguImageTransformer2DModel.from_pretrained(
    "WaveCut/Boogu-Image-0.1-Turbo-OrbitQuant-W4A8", subfolder="transformer", torch_dtype=torch.bfloat16
)
pipe = BooguImageTurboPipeline(
    transformer=transformer,
    vae=AutoencoderKL.from_pretrained(f"{upstream}/vae", torch_dtype=torch.bfloat16),
    scheduler=FlowMatchEulerDiscreteScheduler.from_pretrained(f"{upstream}/scheduler"),
    mllm=AutoModelForImageTextToText.from_pretrained(
        f"{upstream}/mllm", torch_dtype=torch.bfloat16, trust_remote_code=True
    ),
    processor=AutoProcessor.from_pretrained(f"{upstream}/processor", trust_remote_code=True),
).to(device)

with torch.inference_mode():
    image = pipe(
        instruction=["A precise studio photograph of a glass lamp on a dark table"],
        negative_instruction="",
        empty_instruction="",
        height=1024,
        width=1024,
        num_inference_steps=4,
        text_guidance_scale=1.0,
        image_guidance_scale=1.0,
        empty_instruction_guidance_scale=0.0,
        use_dmd_student_inference=True,
        dmd_conditioning_sigma=0.001,
        generator=torch.Generator(device).manual_seed(42),
    ).images[0]

Reference images are passed the way upstream Boogu does (input_images, input_image_paths, align_res); the transformer handles the extra reference tokens unchanged.

Speed and memory

RTX 4060 Ti 16 GB, torch 2.10.0+cu130, BF16, 4 steps with DMD student inference, the MLLM and the transformer + VAE uploaded to the GPU in turn (zero-copy offload), hot runs. The baseline is the SDNQ UINT4 transformer in the same pipeline.

Request	SDNQ UINT4 transformer	OrbitQuant W4A8 transformer	NVML peak, SDNQ → W4A8
1024×1024 text-to-image	8.49 s	5.38 s	6766 → 5982 MiB
704×1280 text-to-image	7.26 s	4.77 s	6606 → 5882 MiB
1024×1024, long prompt	8.75 s	5.80 s	6728 → 6002 MiB
1024×1024, one reference image	20.31 s	10.48 s	7764 → 6724 MiB
1024×1024, two reference images	35.81 s	16.60 s	9044 → 7424 MiB

Times are the transformer's four denoising steps. The MLLM encode (2–4 s) and the VAE decode (0.6–0.9 s) are the same for both transformers. The first request after a process start also compiles the Triton kernels for its shapes.

Quality

Six prompts (photograph, night street, technical diagram, portrait, poster and Russian lettering) at the same seed, against the BF16 transformer: mean PSNR / SSIM 15.21 dB / 0.578 for W4A8 and 14.45 dB / 0.571 for SDNQ UINT4. Four-step DMD sampling moves composition between any two variants, so these numbers describe trajectory drift more than image quality; the matrix above shows the images.

Files

transformer/: config.json (with the fused layout in quantization_config) and the fused weights.
assets/: the BF16 / SDNQ / W4A8 comparison matrix.
benchmark/requests.json: the requests, the protocol and the per-request timings and memory of both transformers.
LICENSE: the Apache License 2.0.
SHA256SUMS: checksums of every file.

License

Apache-2.0, as the upstream model. This is a modified derivative of Boogu Image 0.1 Turbo (quantized transformer weights); no endorsement by the Boogu authors is implied.