1bit-MONSTER/gemma-2-2b-it-GGUF

gemma-2-2b-it — GGUF (Q4_K_M)

bartowski's Q4_K_M GGUF, re-hosted with measured performance for the 1bit engine on Strix Halo.

Licensed under Google's Gemma Terms of Use.

Contents

• gemma-2-2b-it-Q4_K_M.gguf

Measured performance (Strix Halo, Vulkan)

pp512: 2429 tok/s · tg128: 39.4 tok/s

Running it

1bit serve -m gemma-2-2b-it-Q4_K_M.gguf --device vulkan

Attribution

• Base model: google/gemma-2-2b-it.
• Quantization: bartowski/gemma-2-2b-it-GGUF.