pottokao/Qwen-Image-2.1-Text-Encoder-Heretic-W4A8

Qwen-Image-2.1 Text Encoder (Heretic) — W4A8 INT8

Not affiliated with, or endorsed by, Alibaba / Qwen. Community derivative (refusal-ablated) of Qwen/Qwen3-VL-8B-Instruct — the model Qwen-Image-2.1 uses, unmodified, as its text encoder. Qwen releases that model under Apache-2.0, so this derivative is redistributed under Apache-2.0 (see LICENSE and NOTICE).

Asymmetric W4A8 INT8 build of Qwen-Image-2.1-Text-Encoder-Heretic, the abliterated text encoder of Qwen/Qwen-Image-2.1.

File: qwen3vl_8b_w4a8_heretic.safetensors — 5.88 GB (from 16.33 GB bf16)

Comfy-Org ships this format too, so it loads with the stock CLIPLoader. This is the smallest safetensors build here (6 GB). If you just want the exact format the official ComfyUI templates load by default — the "swap the filename and it works" option — use the int8-convrot build instead; this W4A8 is the pick when you want a smaller file.

Ablation (inherited from the bf16 source)

	Refusals	KL divergence
Stock Qwen-Image-2.1 text encoder	100/100	0 (by definition)
This family	5/100	0.0220

Produced with Heretic directional ablation (o_proj + down_proj), 200 trials / 60 startup trials, knee point of the Pareto front. Measured on mlabonne/harmful_behaviors (refusals) and mlabonne/harmless_alpaca (KL). Independently re-checked on the bf16 source: 0/20 refusals, 4/4 benign questions answered correctly.

Full methodology, Pareto table and reproduction command are in the bf16 repo.

Who this is for

Any CUDA GPU, when you want the smallest file. 4-bit weights with 8-bit activations, plus convolution rotation and a Lloyd-Max codebook. Comfy-Org ships this format too, but their templates default to int8-convrot — if you want a drop-in that matches any tutorial screenshot, prefer that one.

Not accelerated on Apple Silicon (CUDA kernels); on a Mac use the GGUF build.

What is quantized, what is protected

Recipe decoded from Comfy-Org's own qwen3vl_8b_w4a8.safetensors and reproduced exactly — the same protection scheme is used for every format in this family:

Layers	Count	Precision
FFN + attention projections	252	4-bit (this repo's format)
embed_tokens, lm_head	2	INT8, per-channel + convrot
Vision tower	351 tensors	bf16 — untouched
norms / biases	—	bf16

79.2 % of parameters go to 4-bit, 14.2 % stay at 8-bit, 6.6 % stay at bf16.

Format details

Tensor	dtype	Note
…weight	int8	4-bit values packed [N, K/2]
…weight_s_rel	float8_e4m3fn	per-group scale
…weight_s_channel	float32	per-channel scale
…weight_codebook	—	Lloyd-Max codebook
…comfy_quant	uint8	per-layer config as JSON bytes

Verified against the official build, byte for byte

The same pipeline was run on the stock (non-ablated) encoder and compared tensor-by-tensor with Comfy-Org's qwen3vl_8b_w4a8.safetensors:

• 4-bit main path, weight_s_rel, weight_s_channel, weight_codebook, vision tower — byte-identical
• the two INT8 layers differ in the last bit only, from a comfy_kitchen version difference (lm_head is unused when the model serves as a text encoder)

So this build is not "hopefully equivalent" to the official recipe — it is the official recipe, applied to abliterated weights.

Usage (ComfyUI)

Place in ComfyUI/models/text_encoders/, then:

CLIPLoader → type: qwen_image → TextEncodeQwenImage21

Requires a ComfyUI build with QwenImage21 support (merged after 2026-09-14). Qwen-Image-2.1 is a full (non-distilled) model — use ~20–30 steps for good quality (few-step / turbo settings will look soft).

Quantization does not fight the ablation

NVFP4 round-trip error measured per layer group:

	Relative error
Ablated layers (o_proj, down_proj)	9.52 %
Untouched layers (q/k/up/gate_proj)	9.51 %
Stock (non-ablated) encoder	9.44 %

Directional ablation is a rank-1 edit; it does not create outliers, so the same recipe applies to ablated and stock weights alike.

Pitfalls found while building this

Each of these produces a valid-looking file that is silently wrong — same size, same tensor count, same format strings. They were only caught by diffing against the official release, or by actually running the model.

1. The vision tower must be excluded. Comfy-Org leaves all 351 vision tensors in bf16; a naive "quantize every 2-D weight" pass eats them.
2. int8_tensorwise needs convrot=True explicitly. The 4-bit path applies convolution rotation internally; the INT8 path defaults to False.
3. comfy_quant must serialize the whole per-layer config, not just {"format": …}, or convrot / convrot_groupsize are dropped.
4. MXFP8 scales must be stored as uint8. TensorCoreMXFP8Layout.quantize() returns float8_e8m0, which ComfyUI's safetensors loader cannot parse (KeyError: 'F8_E8M0').
5. Comfy-Org's repack strips the model.language_model. prefix. Quantizing straight from the HF layout yields keys ComfyUI never finds — the model loads "successfully" and emits noise.

The rest of this family

Repo	What it is
Qwen-Image-2.1-Text-Encoder-Heretic	bf16 source — full precision, 17 GB
Qwen-Image-2.1-Text-Encoder-Heretic-int8-convrot	INT8 convrot — official template default, 8.71 GB
Qwen-Image-2.1-Text-Encoder-Heretic-W4A8	Asymmetric W4A8 INT8 — smallest CUDA build, 5.88 GB ← you are here
Qwen-Image-2.1-Text-Encoder-Heretic-NVFP4	NVFP4 (w4) for Blackwell — native FP4 tensor cores, 5.87 GB
Qwen-Image-2.1-Text-Encoder-Heretic-GGUF	GGUF Q4_K_M + mmproj — ComfyUI via CLIPLoaderGGUF + ComfyUI-GGUF-Qwen3VL-TE add-on, 4.68 GB