0hmn/gemma-4-e2b-it-tessera-mlx

Gemma 4 E2B IT, measured mixed-precision MLX quantization (text only)

A text-only MLX quantization of google/gemma-4-E2B-it made for Tessera's on-device sensitive-span proposer. Every weight table's precision was chosen from measured damage: the BF16 file was the only weight input, and no quantization tool, importance matrix or published recipe decided anything. It uses MLX's stock affine format (group size 64, 4-bit default with 152 per-layer overrides from 3 to 8 bits), so mlx-lm and mlx-swift-lm load it unmodified.

Measured against BF16

Checkpoint	Weights	KL vs BF16 (set 1)	KL vs BF16 (set 2)	Same top word (set 2)	Perplexity (set 2)
stock uniform 8-bit	4.92 GB	0.005	0.005	98.1%	3.92
stock uniform 6-bit	3.76 GB	0.033	0.033	95.1%	3.76
this checkpoint	2.30 GB	0.087	0.095	91.6%	4.15
mlx-community/gemma-4-e2b-it-4bit text weights (stock 4-bit)	2.60 GB	0.449	0.452	81.0%	5.75

BF16 perplexity on set 2: 3.91. Set 1: 22,566 positions written by the stock 4-bit model; set 2: 23,129 positions written by BF16; 16 document kinds never used for tuning. KL is against BF16 computed in float32; candidates run with bf16 activations, as deployed.

Speed and memory (mlx-lm 0.32.2, Apple M3 Max)

Tessera's real request: shipped system prompt plus a 512-token document chunk (697 prompt tokens), then 128 generated tokens; median of 3 runs.

Checkpoint	Prompt tok/s	Generate tok/s	Peak memory
this checkpoint	4,430	111.5	3.04 GB
mlx-community/gemma-4-e2b-it-4bit text weights (stock 4-bit)	4,139	113.1	3.35 GB

How it was made

1. The float model collected each table's input second moment H over 68.8k tokens of fictional professional documents, as plain text and as Tessera's span-proposal requests.
2. Rounding minimised the output error eᵀHe, not the weight error: weighted least squares per group, bfloat16-snapped scales and biases, then integer coordinate descent on codes and scales.
3. Each of the 278 tables was rounded alone and its damage measured as KL against BF16 in float32 activations (with bf16 activations this model is chaotic at the 1e-3 KL level).
4. A multiple-choice knapsack spent the byte budget across 2/3/4/5/6/8-bit tables to minimise total damage.

Changes from the original

• Weights quantized as described; vision and audio towers removed (text generation only).
• tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json and processor_config.json are copied from mlx-community/gemma-4-e2b-it-4bit at 238767527555cb75a05732a84dff5d6ba0dd6809 so prompts render as in Tessera's previous pin.

Provenance

• Source: google/gemma-4-E2B-it at 4adf15748de9e1a0097b6274cee9c15836c41c37, model.safetensors SHA-256 2db5482b20d746879bb3ef79b5203e9075a2e2b98f54ec7c2f281c1477ddc550.
• Published as 0hmn/gemma-4-e2b-it-tessera-mlx.

File	Bytes	SHA-256
chat_template.jinja	17,336	2f1b4d75d067bae3fe44e676721c7f077d243bc007156cb9c2f8b5836613d082
config.json	24,906	90b46c435576db3a3e5d313169c06e8a2a5ff8b898ef2e6a79e68b492aed9554
generation_config.json	208	d4226bbe3117d2d253ba4609720ba82c6c4ce4627a9a6ae05387c78983ac03de
model.safetensors	2,299,021,182	07963e8d9928dcc3da6ae3f0090771c5637544d0ccfaa5d4580e13b0092da624
model.safetensors.index.json	91,141	f3192f8ab143b1c520cae444b54a41322fdad15bf8cfad4100b1c029a0345b68
processor_config.json	1,316	de3e580aebdc98272d4c4547daffe6525fcbae18a83a0e0bcf0d7444d4ee6f37
tokenizer.json	32,169,626	cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
tokenizer_config.json	2,740	080d9e1aff284e2f6043889cd05367966f7c7b80e025fbc0b06745e218158656

Limitations

• Text only. Quality was measured on synthetic fictional documents and on Tessera's 76-span evaluation corpus; it is not a general benchmark.
• Greedy decoding can still differ from BF16 on individual spans.

License

Gemma 4 is provided by Google under the Apache License 2.0; this derivative is distributed under the same license (see LICENSE).