mrysh/qwen3.5-4b-fp8

qwen3.5-4b - fp8

Quantized version of Qwen/Qwen3.5-4B made for a quantization benchmark (4 models x 8 techniques, evaluated with vLLM on BFCL, Energy-QA, hiring-bias and MedQA).

• method: fp8, about 8 bits, group size None
• library: nvidia-modelopt 0.47.0
• calibration samples: 512
• size on disk: 4.866 GB

FP8 (llm-compressor / ModelOpt format), loads in vLLM.

Evaluated as a text model (vLLM, thinking off). Same license as the base model; see the base model card.