Clef-Flash-AWQ

4-bit AWQ quantization of Clef-Flash, a 9B-parameter multimodal model post-trained from Qwen3.5-9B.

Quantization

• Scheme: W4A16_ASYM — int4 weights, group size 128, asymmetric int8 zero-points
• Format: compressed-tensors pack-quantized (vLLM-native)
• Quantized: all text Linear layers (full-attention qkv/o, GDN projections, MLP)
• Excluded (kept bf16): lm_head, all visual-tower weights, GDN in_proj_a/in_proj_b
• See recipe.yaml and QUANTIZATION_MANIFEST.json for the full recipe

The model runs at roughly half the memory of the bf16 original with negligible quality loss for its decision/chat workloads.

Usage (vLLM)

vllm serve cryowastakenbycryo/Clef-Flash-AWQ \
    --tensor-parallel-size 2 \
    --trust-remote-code

Standard vLLM + compressed-tensors; no custom code required for the backbone.

Notes

• The joint decision head used by the SystemOne serving path is not part of this repo — it is a separate artifact deployed alongside the backbone.
• Multimodal: image/video inputs supported per the base model's processor.