mrysh/gemma-4-e4b-it-gptq_4bit

gemma-4-e4b-it - gptq_4bit

Quantized version of google/gemma-4-E4B-it made for a quantization benchmark (4 models x 8 techniques, evaluated with vLLM on BFCL, Energy-QA, hiring-bias and MedQA).

• method: gptq, about 4 bits, group size 128
• library: gptqmodel 7.6.0
• calibration samples: 512
• size on disk: 10.079 GB

GPTQ, loads in vLLM (vllm serve <repo>).

Evaluated as a text model (vLLM, thinking off). Same license as the base model; see the base model card.