FLUX.2-klein-base-9B — OpenVINO INT4
OpenVINO IR (INT4) conversion of the FLUX.2-klein 9B base architecture (rectified-flow transformer + Qwen3 text encoder + VAE). Source weights: community full-pipeline mirror (official BFL repo is license-gated); architecture verified standard diffusers layout before conversion.
Why OpenVINO?
- Runs on any CPU with no CUDA/NVIDIA dependency, plus GPU acceleration where available (AMD, Nvidia, Intel) — no vendor-locked passes used
- Transformer is 4.9GB INT4 — the per-inference hot path fits 8GB VRAM cards
Deployment note (8GB cards)
Total repo is 9.1GB because the Qwen3 text encoder (8.5B params) has a
~4.1GB INT4 floor. Standard practice, reflected in the usage below: keep the
transformer on GPU, text encoder + VAE on CPU. The text encoder runs once
per prompt, so this costs no per-step speed.
Conversion
| Parameter | Value |
|---|---|
| Tool | optimum-cli export openvino --weight-format int4 --group-size 64 --all-layers (device-neutral) |
| Precision | INT4 transformer + text encoder, FP topologies otherwise |
| Verified | transformer, text_encoder, vae_encoder, vae_decoder all load in ov.Core on Kaggle GPU |
Usage (any device)
import openvino as ov
core = ov.Core()
# Hot path on GPU, one-shot modules on CPU:
trafo = core.compile_model(core.read_model("transformer/openvino_model.xml"), "GPU")
te = core.compile_model(core.read_model("text_encoder/openvino_model.xml"), "CPU")
vae = core.compile_model(core.read_model("vae_decoder/openvino_model.xml"), "CPU")