---
base_model: google/gemma-2-9b-it
base_model_relation: quantized
inference: false
license: gemma
model_name: Gemma-2-9B-it-4Bit-GPTQ
pipeline_tag: text-generation
quantized_by: qilowoq
tags:
- gptq
language:
- en
- ru
---

# Gemma-2-2B-it-4Bit-GPTQ

- Original Model: [gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it)
- Model Creator: [google](https://huggingface.co/google)

## Quantization

- This model was quantized with the Auto-GPTQ library and dataset containing english and russian wikipedia articles. It has lower perplexity on russian data then other GPTQ models.

| Model                                                                                                 |          bits | Perplexity (russian wiki) |
| ----------------------------------------------------------------------------------------------------- | ------------- | ------------------------- |
[gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it)                                            | 16bit         | 6.2152                    |
[Granther/Gemma-2-9B-Instruct-4Bit-GPTQ](https://huggingface.co/Granther/Gemma-2-9B-Instruct-4Bit-GPTQ) | 4bit          | 6.4966                    |
this model                                                                                              | 4bit          | 6.3593                    |