MeghanaMakkapati13/gemma_quantization

Gemma-4 E4B IQ4_XS — Quantized GGUF

Compressed submission for the Resilient AI Challenge (Image-to-Text category), joint initiative of the Government of France, Government of India, UNESCO, and ITU.

Model details

Field	Value
Base model	google/gemma-4-E4B-IT
Fine-tuning	Parameter-efficient fine-tuning
Compression	IQ4_XS (~4.25 bpw, importance-matrix quantization)
Format	GGUF (llama.cpp)
Vision projector	mmproj-BF16.gguf (BF16, unchanged)

Model size

Model	Size
F16 baseline	15.05 GB
IQ4_XS (this model)	5.06 GB
mmproj (vision projector)	0.99 GB
Compression ratio	~3.7× smaller than F16

Running the model

Use llama-server with the provided llama_server_config.json:

llama-server \
  -m gemma4-E4B-IQ4_XS.gguf \
  --mmproj mmproj-BF16.gguf \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 99 \
  --ctx-size 8192

Generation parameters: temperature=1.0, top_p=0.95, top_k=64