mrysh/gemma-4-e4b-it-bnb_4bit

gemma-4-e4b-it - bnb_4bit

Quantized version of google/gemma-4-E4B-it made for a quantization benchmark (4 models x 8 techniques, evaluated with vLLM on BFCL, Energy-QA, hiring-bias and MedQA).

• method: bnb, about 4 bits, group size None
• library: bitsandbytes 0.50.2
• calibration samples: None
• size on disk: 9.54 GB

bitsandbytes (in-flight quantized weights), loads in vLLM with the vllm-bnb-plugin.

Evaluated as a text model (vLLM, thinking off). Same license as the base model; see the base model card.