pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-int8-convrot

🍱 All-GGUF Qwen-Image-2.1: text encoder GGUF + DiT GGUF (Q8_0 / Q6_K / Q4_K_M, loads in stock ComfyUI-GGUF) + PE-T2I GGUF.

Qwen-Image-2.1 Text Encoder (Heretic) — INT8 convrot

Not affiliated with, or endorsed by, Alibaba / Qwen. Community derivative (refusal-ablated) of Qwen/Qwen3-VL-8B-Instruct — the model Qwen-Image-2.1 uses, unmodified, as its text encoder. Qwen releases that model under Apache-2.0, so this derivative is redistributed under Apache-2.0 (see LICENSE and NOTICE).

INT8-convrot build of Qwen-Image-2.1-Text-Encoder-Heretic, the abliterated text encoder of Qwen/Qwen-Image-2.1.

File: qwen3vl_8b_int8_convrot_heretic.safetensors — 8.71 GB (from 16.33 GB bf16)

This is the format Comfy-Org's official Qwen-Image-2.1 templates use by default (qwen3vl_8b_int8_convrot.safetensors). If you followed any official / community Qwen-Image-2.1 tutorial, this is a drop-in replacement — just swap the filename. It keeps every projection at INT8 (heavier than the 4-bit W4A8 build, but the most faithful to the stock template).

Ablation (inherited from the bf16 source)

	Refusals	KL divergence
Stock Qwen-Image-2.1 text encoder	100/100	0 (by definition)
This family	5/100	0.0220

Produced with Heretic directional ablation (o_proj + down_proj), 200 trials / 60 startup trials, knee point of the Pareto front. Measured on mlabonne/harmful_behaviors (refusals) and mlabonne/harmless_alpaca (KL). Independently re-checked on the bf16 source: 0/20 refusals, 4/4 benign questions answered correctly.

Full methodology, Pareto table and reproduction command are in the bf16 repo.

Who this is for

Any CUDA GPU running the official Qwen-Image-2.1 workflow. This is the format the ComfyUI templates ship with, so it is the least-surprising choice — no format mismatch, no special nodes.

Not accelerated on Apple Silicon (CUDA kernels); on a Mac the GGUF build has real Metal support. Want a smaller CUDA file? The W4A8 build is 6.3 GB.

What is quantized, what is protected

Recipe decoded from Comfy-Org's own qwen3vl_8b_int8_convrot.safetensors and reproduced exactly:

Layers	Count	Precision
FFN + attention projections	252	INT8 (per-channel + convrot)
embed_tokens, lm_head	2	INT8 (per-channel + convrot)
Vision tower	351 tensors	bf16 — untouched
norms / biases	—	bf16

Every projection (254 tensors) is INT8 tensorwise with convolution rotation (convrot_groupsize 256, per-channel scale [out, 1]); only the vision tower and norms stay bf16.

Format details

Tensor	dtype	Note
…weight	int8	rotated INT8 weights
…weight_scale	per-channel	shape [out, 1]
…comfy_quant	uint8	{"format":"int8_tensorwise","convrot":true,"convrot_groupsize":256}

Verified against the official build

The per-layer spec was decoded from Comfy-Org's qwen3vl_8b_int8_convrot.safetensors and reproduced on the abliterated weights, then compared tensor-by-tensor with the official file (2026-09-23):

• key set identical — 1258 tensors, zero shape mismatches
• every comfy_quant config matches (int8_tensorwise / convrot / convrot_groupsize 256)
• vision tower stays bf16, same as the official build

And end-to-end in ComfyUI's official Qwen-Image-2.1 workflow it generates images normally — verified by actually rendering, not just loading.

Usage (ComfyUI)

Place in ComfyUI/models/text_encoders/, then:

CLIPLoader → type: qwen_image → TextEncodeQwenImage21

Requires a ComfyUI build with QwenImage21 support (merged after 2026-09-14). Qwen-Image-2.1 is a full (non-distilled) model — use ~20–30 steps for good quality (few-step / turbo settings will look soft).

Pitfalls found while building this

Each of these produces a valid-looking file that is silently wrong — same size, same tensor count, same format strings. They were only caught by diffing against the official release, or by actually running the model.

1. The vision tower must be excluded. Comfy-Org leaves all 351 vision tensors in bf16; a naive "quantize every 2-D weight" pass eats them.
2. int8_tensorwise needs convrot=True explicitly. The INT8 path defaults to False, so convolution rotation is silently skipped.
3. comfy_quant must serialize the whole per-layer config, not just {"format": …}, or convrot / convrot_groupsize are dropped.
4. Comfy-Org's repack strips the model.language_model. prefix. Quantizing straight from the HF layout yields keys ComfyUI never finds — the model loads "successfully" and emits noise. This build is remapped to the ComfyUI layout (verified: key set identical to the official file).

The rest of this family

Repo	What it is
Qwen-Image-2.1-Text-Encoder-Heretic	bf16 source — full precision, 17 GB
…-int8-convrot	INT8 convrot — official template default, 8.71 GB ← you are here
…-W4A8	Asymmetric W4A8 INT8 — smaller CUDA build, 5.88 GB
…-NVFP4	NVFP4 (w4) for Blackwell — native FP4 tensor cores, 5.87 GB
…-GGUF	GGUF Q4_K_M + mmproj — ComfyUI via CLIPLoaderGGUF + ComfyUI-GGUF-Qwen3VL-TE add-on, 4.68 GB