DreamFast/Qwen3-VL-4b-Heretic-ComfyUI

Qwen3-VL-4B-Instruct Heretic (ComfyUI)

ComfyUI checkpoints of the Heretic abliteration of Qwen3-VL-4B-Instruct. Five quantisations are included, covering everything from any modern GPU through to Blackwell, suitable as an uncensored text encoder for image-generation workflows including Krea 2, which runs on a Qwen3VL-4B encoder.

Available formats

This model is published across three repos. Pick the one that matches your runtime.

Repo	Best for	Contents
Qwen3-VL-4b-Heretic	transformers, vLLM, HF Hub	bf16 weights with config, vision encoder preserved
Qwen3-VL-4b-Heretic-GGUF	llama.cpp, Ollama, LM Studio, ComfyUI-GGUF	GGUF quants from Q3_K_M up to F16 (text path)
Qwen3-VL-4b-Heretic-ComfyUI (this repo)	ComfyUI text encoder	bf16, fp8, int8, nvfp4 and mxfp8 checkpoints

The Docker setup, scripts and configs that produced these files are in Heretic Docker.

Why this variant?

Several Heretic trials were run against Qwen3-VL-4B and all of them reach 100% HarmBench ASR, up from 30.8% on the base. This build was picked because, with safety tied, it wins on the tie-breakers:

	Base	Heretic
HarmBench ASR	30.8%	100%
KL divergence (lower is better)		0.0283 (lowest of the candidates)
GSM8K	78.62%	77.18% (−1.83%, smallest drop)
MMLU	69.58%	69.61% (+0.03%)
Tensors changed		54 (pure rank-1)

See the full report for the comparison.

Files

File	Format	Size	HW	Description
qwen3-vl-4b-heretic.safetensors	bf16	8.3 GB	Any	Full precision
qwen3-vl-4b-heretic_fp8_e4m3fn.safetensors	FP8 E4M3	4.2 GB	Ada+	Per-tensor scaled
qwen3-vl-4b-heretic_int8.safetensors	INT8	4.5 GB	Any	ConvRot learned rounding, block-wise
qwen3-vl-4b-heretic_nvfp4.safetensors	NVFP4 E2M1	2.9 GB	Blackwell*	4-bit float, double quantisation
qwen3-vl-4b-heretic_mxfp8.safetensors	MXFP8	4.7 GB	Blackwell	Microscaling FP8, E8M0 block scales

* NVFP4 also runs on older GPUs through software dequantisation (tested on an RTX 4090). Native FP4 tensor cores need SM100+ (RTX 5090/5080).

Which quant should I pick?

• bf16 (8.3 GB) is full precision. Use it when you have the VRAM and want zero quantisation loss.
• INT8 / ConvRot (4.5 GB, any GPU) is the recommended pick. Learned rounding keeps it near-lossless, and it runs on any Ampere+ GPU without needing Blackwell.
• FP8 E4M3 (4.2 GB, Ada+) is fast and small, a good fit for RTX 4090 / RTX 4500 and up.
• NVFP4 (2.9 GB) is the smallest. Native FP4 on Blackwell, with software dequant on older GPUs.
• MXFP8 (4.7 GB, Blackwell) handles dynamic range better than per-tensor FP8 thanks to E8M0 block scales.

Usage

1. Download one of the files above.
2. Place it in ComfyUI/models/text_encoders/.
3. Use the matching loader node in your workflow.

With Krea 2

Krea 2 is an image-generation model that runs on a Qwen3VL-4B text encoder. Because these checkpoints are the same Qwen3-VL-4B architecture (abliterated), you can drop one in as the encoder in a Krea 2 ComfyUI workflow to give it an uncensored text encoder. The fp8 checkpoint is the closest match to the stock qwen3vl_4b_fp8_scaled.safetensors; use int8 or bf16 if you want higher fidelity and have the VRAM.

Quantisation format details

FP8 (E4M3) is per-tensor scaled quantisation via convert-to-quant. It runs on Ada (RTX 4090) and newer, giving a good balance of size and quality.

INT8 (ConvRot) is block-wise symmetric INT8 with ConvRot learned rounding. For each weight tensor, an SVD-guided gradient descent loop (Prodigy optimiser) learns the rounding direction that minimises output error, which keeps INT8 quality near lossless. It runs on any modern GPU (Ampere+) with no Blackwell requirement. Vision encoder weights may be left out because of their non-standard tensor dimensions.

NVFP4 (E2M1) is 4-bit floating point with double quantisation (a per-tensor f32 scale plus a per-block FP8 scale, block size 16). It is about three times smaller than bf16 and loads natively in ComfyUI without plugins. Blackwell GPUs (RTX 5090/5080, SM100+) use native FP4 tensor cores for best speed, while older GPUs fall back to software dequantisation (tested on an RTX 4090).

MXFP8 is microscaling FP8 (the OCP MX standard). It stores FP8 E4M3 data with E8M0 (power-of-two) per-block scales on 32-element blocks, handling dynamic range better than per-tensor FP8. It needs SM100+ (Blackwell).

How these were made

Produced with Heretic Docker, which wraps:

• Heretic for the abliteration
• convert-to-quant for FP8, INT8 (ConvRot) and MXFP8 quantisation
• comfy_kitchen for the NVFP4/MXFP8 CUDA kernels

Limitations

• Inherits every limitation of the base Qwen3-VL-4B-Instruct model.
• Abliteration reduces refusals but does not remove them completely.
• NVFP4 and MXFP8 run best on Blackwell GPUs; NVFP4 also runs on older GPUs through software dequantisation.
• INT8 ConvRot may leave out vision encoder weights because of non-standard tensor dimensions.
• Inherits the abliteration caveats: TruthfulQA and GSM8K sit measurably lower than the base model (see the report).

Acknowledgements

• Qwen team for Qwen3-VL-4B-Instruct
• Heretic by p-e-w, the abliteration tool
• convert-to-quant by silveroxides for INT8 ConvRot, FP8 scaling and MXFP8 quantisation
• comfy_kitchen by Comfy-Org for the NVFP4/MXFP8 CUDA kernels
• Abliterlitics for the variant forensics

Disclaimer

This model has had its safety alignment removed. It complies with harmful requests, including content related to violence, illegal activities and other harmful behaviour. Use it responsibly and in line with the laws and regulations that apply to you. The authors do not condone or encourage using this model for harmful purposes.