tonera/FLUX.2-klein-4B-Nunchaku

Model Card (SVDQuant / Nunchaku)

Language: English | 中文

Model and upstream

• Quantized weights repo: tonera/FLUX.2-klein-4B-Nunchaku
• Official full-precision source: black-forest-labs/FLUX.2-klein-4B
• Quantized Transformer in this repo: svdq-<precision>_r32-FLUX.2-klein-4B-Nunchaku.safetensors; use nunchaku.utils.get_precision() for <precision> (commonly fp4 or int4) so the file name matches your environment and Nunchaku build

Option 1 (for users comfortable with coding)

• Engine: vitoom-nunchaku — community-maintained Nunchaku build with FLUX.2 Klein support (FP4/INT4 SVDQuant inference)
• Framework: Diffusers with Flux2KleinPipeline support (official examples install from source):

pip install "git+https://github.com/huggingface/diffusers.git"

Install vitoom-nunchaku (recommended)

Upstream Nunchaku has not merged FLUX.2 Klein support for a long time (PR #926 still pending). Do not copy patch files manually. Install the prebuilt wheel that matches your platform, Python, and CUDA:

1. Install matching PyTorch 2.11 (cu128 or cu130) first.
2. Download and install a wheel from tonera/vitoom-nunchaku.

Example (x86_64, Python 3.11, CUDA 13.0):

pip install torch==2.11.* torchvision==0.26.* torchaudio==2.11.* \
  --index-url https://download.pytorch.org/whl/cu130

hf download tonera/vitoom-nunchaku \
  nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl \
  --local-dir ./wheels

pip install ./wheels/nunchaku-1.3.0.dev20260622+cu13.0torch2.11-cp311-cp311-linux_x86_64.whl

For other platforms (cu128, cp310, ARM64 aarch64), see the wheel table in the vitoom-nunchaku README.

Verify:

python -c "import nunchaku; from nunchaku import NunchakuFlux2Transformer2DModel; print(nunchaku.__version__)"

Minimal example (image-to-image + quantized Transformer)

Assumes vitoom-nunchaku is installed and weights are available locally or at tonera/FLUX.2-klein-4B-Nunchaku; set REPO to your directory or Hugging Face model id.

import torch
from diffusers import Flux2KleinPipeline
from diffusers.utils import load_image

from nunchaku import NunchakuFlux2Transformer2DModel
from nunchaku.utils import get_precision

REPO = "tonera/FLUX.2-klein-4B-Nunchaku"  # or local absolute path
NAME = "FLUX.2-klein-4B-Nunchaku"

transformer = NunchakuFlux2Transformer2DModel.from_pretrained(
    f"{REPO}/svdq-{get_precision()}_r32-{NAME}.safetensors",
    torch_dtype=torch.bfloat16,
)
pipe = Flux2KleinPipeline.from_pretrained(
    REPO, torch_dtype=torch.bfloat16, transformer=transformer
)

pipe.to("cuda")

ref = load_image("https://example.com/your_ref.png").convert("RGB")
image = pipe(
    prompt="Describe your edit in English…",
    image=ref,
    guidance_scale=1.0,  # matches official Klein examples; tune if needed
    num_inference_steps=4,  # common for the distilled model; see Diffusers docs otherwise
    generator=torch.Generator("cpu").manual_seed(1),
).images[0]
image.save("flux2_klein_4b_nunchaku.png")

For text-to-image, omit image (behavior per current Diffusers Flux2KleinPipeline docs). Use pipe.enable_model_cpu_offload() or similar if VRAM is tight.

Recommended inference settings

Parameter	Suggested value	Notes
num_inference_steps	4	Common for distilled Klein models
guidance_scale	1.0	Matches official Klein examples

If the steps above feel too difficult, install the vitoom platform instead (see below).

Option 2 (recommended: vitoom)

We recommend deploying and running via vitoom: it ships a complete vitoom-nunchaku runtime, Web UI, multi-reference editing, and LoRA management—no manual wheel install or file copying. See the vitoom repo and docker-usage-en.md for setup.

1. Prepare the environment

You need Docker, Docker Compose, and an NVIDIA GPU for inference (driver must support CUDA 13.0, matching vitoom’s cu130 inference images). Confirm GPU access:

docker run --rm --gpus all nvidia/cuda:13.0.0-base-ubuntu24.04 nvidia-smi

2. Generate config and pull images

In the vitoom repo root, run the setup wizard (recommended):

git clone https://github.com/tonera/vitoom.git
cd vitoom
python scripts/setup_vitoom.py

Or copy .env.example to .env, fill in required fields (VITOOM_BACKEND_URL, VITOOM_WS_URL, VITOOM_INFERENCE_UPLOAD_AUTH_SECRET, etc.), then pull images:

python scripts/load_vitoom_images.py --components backend,visual

3. Start services

Start Backend:

docker compose up -d backend

Open http://127.0.0.1:8888 in your browser (port follows VITOOM_SERVER_PORT in .env).

Start the Visual inference service (FLUX.2 Klein is an image model; this profile is required):

docker compose -f docker-compose.inference.release.yml --profile visual up -d

Optional: run python scripts/download_initial_models.py to batch-download common models; or place weights under the host resources/models directory (override via VITOOM_MODELS_HOST_DIR in .env).

4. Use this model in the Web UI

1. Log in to the vitoom Web UI, open Models, download and activate tonera/FLUX.2-klein-4B-Nunchaku
2. Open the Image workspace, select the activated FLUX.2 Klein 4B model, and run text-to-image or image editing (multi-reference editing supported)

You can also run the Python example above directly inside the vitoom Visual inference container.

LoRA

FLUX.2 Klein supports LoRA on the quantized Transformer when using vitoom-nunchaku (wheel or vitoom Visual image)—no manual file copying required:

lora_path = "/path/to/your_lora.safetensors"
transformer.update_lora_params(lora_path)
transformer.set_lora_strength(0.8)

License and compliance

These quantized weights are derived from FLUX.2-klein-4B. The upstream model is under Apache 2.0; confirm upstream license terms and Black Forest Labs’ acceptable use policy before commercial use or redistribution.