QmogAI/gemma4-e2b-int8

Gemma 4 E2B int8 for gemma4-e2b.c

This is the ready-to-run model file for gemma4-e2b.c, a small educational implementation of Gemma 4 E2B inference on CPU.

The project's exporter.py converts the original Google Gemma 4 E2B QAT checkpoint into one 5.2 GiB file containing the model configuration, tokenizer, and weights. During export, the weights are quantized to int8.

Download

hf download QmogAI/gemma4-e2b-int8 gemma4-E2B-int8.bin --local-dir .

Run

git clone https://github.com/ryanssenn/gemma4-e2b.c
cd gemma4-e2b.c
make
./run -m ./gemma4-E2B-int8.bin -t 1.0 -n 256 "Why is the sky blue?"

Export your own

If you would rather create the file from the original checkpoint:

python3 -m pip install -r requirements.txt
python3 exporter.py /path/to/gemma-4-E2B-it-qat-q4_0-unquantized -o ./gemma4-E2B-int8.bin

The runtime is made for x86-64 CPUs with AVX2 and is intended for learning and experimentation.