rfqckpt-main

Quantized and rotated checkpoints of Qwen2.5-VL-7B-Instruct, InternVL3-9B and Molmo2-O-7B. Access is gated and granted manually.

Layout is <model>/<family>/<variant>/; every leaf is a complete checkpoint folder.

family	contents
quant	bitsandbytes fp4 / nf4 applied to all linears, the language model only, or the vision tower only; GPTQ and AWQ (language model) via llm-compressor
rotation	Qwen fp4_all with a rotation applied, stored as bf16 fake-quant weights
rotation_rq	the same rotations re-quantized to bitsandbytes fp4
mixed_c3	Qwen fp4_all with the top-12 selected linears (per criterion) kept in bf16

Every rotation, rotation_rq and mixed_c3 variant is built from qwen2.5-vl-7b/quant/fp4_all. build_info.json in each folder records how it was built and the sha256 of every file. manifest.csv lists every folder.

Download one variant

hf download aabceh049/rfqckpt-main --include "qwen2.5-vl-7b/quant/fp4_all/*" --local-dir ckpts

Load from the local folder (from_pretrained("ckpts/qwen2.5-vl-7b/quant/fp4_all"), with trust_remote_code=True for InternVL and Molmo). subfolder= does not work with remote code.

Checkpoints taken from the Hub (not re-hosted)

model	arm	source	revision
qwen2.5-vl-7b	awq_lang	Qwen/Qwen2.5-VL-7B-Instruct-AWQ	536a35794df8831aa814970ee8f89eff577e7718
qwen2.5-vl-7b	gptq_lang_rh	RedHatAI/Qwen2.5-VL-7B-Instruct-quantized.w4a16	958cdcc6838acf5cba2f46148a3dbda840ea7096
internvl3-9b	awq_lang	OpenGVLab/InternVL3-9B-AWQ	73578c0e169a5a0c1eff174f0f403c0d55605f67

All variants

path	quantization	GB
internvl3-9b/quant/fp4_all	bitsandbytes fp4	6.33
internvl3-9b/quant/fp4_lang	bitsandbytes fp4	6.78
internvl3-9b/quant/fp4_vision	bitsandbytes fp4	17.83
internvl3-9b/quant/gptq_lang	compressed-tensors	6.77
internvl3-9b/quant/nf4_all	bitsandbytes nf4	6.33
internvl3-9b/quant/nf4_lang	bitsandbytes nf4	6.78
internvl3-9b/quant/nf4_vision	bitsandbytes nf4	17.83
molmo2-o-7b/quant/awq_lang	compressed-tensors	5.94
molmo2-o-7b/quant/fp4_all	bitsandbytes fp4	5.24
molmo2-o-7b/quant/fp4_lang	bitsandbytes fp4	5.92
molmo2-o-7b/quant/fp4_vision	bf16_fakequant	15.53
molmo2-o-7b/quant/gptq_lang	compressed-tensors	5.92
molmo2-o-7b/quant/nf4_all	bitsandbytes nf4	5.24
molmo2-o-7b/quant/nf4_lang	bitsandbytes nf4	5.92
molmo2-o-7b/quant/nf4_vision	bf16_fakequant	15.53
qwen2.5-vl-7b/quant/fp4_all	bitsandbytes fp4	5.91
qwen2.5-vl-7b/quant/fp4_lang	bitsandbytes fp4	6.91
qwen2.5-vl-7b/quant/fp4_vision	bitsandbytes fp4	15.6
qwen2.5-vl-7b/quant/nf4_all	bitsandbytes nf4	5.91
qwen2.5-vl-7b/quant/nf4_lang	bitsandbytes nf4	6.91
qwen2.5-vl-7b/quant/nf4_vision	bitsandbytes nf4	15.6
qwen2.5-vl-7b/mixed_c3/absp12	bitsandbytes fp4	7.31
qwen2.5-vl-7b/mixed_c3/errp12	bitsandbytes fp4	7.32
qwen2.5-vl-7b/mixed_c3/fisherp12	bitsandbytes fp4	7.32
qwen2.5-vl-7b/mixed_c3/pmr12_s0	bitsandbytes fp4	7.29
qwen2.5-vl-7b/mixed_c3/ptop12	bitsandbytes fp4	7.13
qwen2.5-vl-7b/mixed_c3/snipp12	bitsandbytes fp4	7.31
qwen2.5-vl-7b/mixed_c3/tacqp12	bitsandbytes fp4	7.31
qwen2.5-vl-7b/mixed_c3/top12	bitsandbytes fp4	7.32
qwen2.5-vl-7b/rotation/identity	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation/keep_cos6	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation/keep_ends6	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation/keep_mid6	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation/keep_notcos6	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation/matched_L24h0	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation/real_L24h0	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation/seq_all	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation_rq/seq_all	bitsandbytes fp4	5.91
qwen2.5-vl-7b/rotation_rq/seq_cos6	bitsandbytes fp4	5.91
qwen2.5-vl-7b/rotation/seq_matched_all	bf16_fakequant	16.6
qwen2.5-vl-7b/rotation_rq/seq_matched_all	bitsandbytes fp4	5.91
qwen2.5-vl-7b/rotation_rq/seq_matched_cos6	bitsandbytes fp4	5.91
qwen2.5-vl-7b/rotation_rq/seq_matched_notcos6	bitsandbytes fp4	5.91
qwen2.5-vl-7b/rotation_rq/seq_matched_six	bitsandbytes fp4	5.91
qwen2.5-vl-7b/rotation_rq/seq_notcos6	bitsandbytes fp4	5.91
qwen2.5-vl-7b/rotation_rq/seq_six	bitsandbytes fp4	5.91