Hugging Face's logo

raymsm
/
gemma-2-9b-it-GGUF

GGUF
conversational
Model card Files
xet
 Community

YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.

gemma-2-9b-it GGUF

Quantized GGUF versions of google/gemma-2-9b-it generated using llama.cpp on an Oracle Cloud VM.

Repositories & Quantization Details

This repository contains the following quantization formats:

Quantization Type File Size Quantization Duration Peak RAM Used Description
Q4_K_M 5.37 GB 105.0s 1.4 MB Medium-sized, good balance of size/perplexity.
Q5_K_M 6.19 GB 78.5s 1.4 MB Medium-sized, good balance of size/perplexity.
Q6_K 7.07 GB 54.8s 1.4 MB Medium-sized, good balance of size/perplexity.
Q8_0 9.15 GB 26.7s 1.4 MB Medium-sized, good balance of size/perplexity.

Sanity Check Prompts & Responses

Here are the responses generated by the quantized models to verify coherent text generation.

Format: Q4_K_M

  • Prompt: What is the capital of France? Answer in one word.
  • Response:
    What is the capital of France? Answer in one word.

Paris [end of text]

- **Prompt**: *Write a short four-line poem about coding in Python.*
- **Response**:
```text
Write a short four-line poem about coding in Python.

Indentation guides the way,
With loops and functions, we play.
From scripts to apps, it takes flight,
Python's power, shining bright. 


Let me know if you'd like to see more poems about coding or any other topic! 
 [end of text]
  • Prompt: Explain why the sky is blue in one simple sentence.
  • Response:
    Explain why the sky is blue in one simple sentence.

Blue light scatters more than other colors in the atmosphere. [end of text]


### Format: Q5_K_M
- **Prompt**: *What is the capital of France? Answer in one word.*
- **Response**:
```text
What is the capital of France? Answer in one word.

Paris [end of text]
  • Prompt: Write a short four-line poem about coding in Python.
  • Response:
    Write a short four-line poem about coding in Python.

With Python's grace, I weave my code, Indentation guides, a structured road. From loops to functions, clear and bright, A symphony of logic, day and night.

[end of text]

- **Prompt**: *Explain why the sky is blue in one simple sentence.*
- **Response**:
```text
Explain why the sky is blue in one simple sentence.

**Blue light is scattered more by the atmosphere than other colors, making it appear dominant in the sky.** 
 [end of text]

Format: Q6_K

  • Prompt: What is the capital of France? Answer in one word.
  • Response:
    What is the capital of France? Answer in one word.

Paris [end of text]

- **Prompt**: *Write a short four-line poem about coding in Python.*
- **Response**:
```text
Write a short four-line poem about coding in Python.

With Python's grace, I weave my code,
A symphony of logic, clear and bold.
From functions swift to classes grand,
A digital world, at my command. 


 [end of text]
  • Prompt: Explain why the sky is blue in one simple sentence.
  • Response:
    Explain why the sky is blue in one simple sentence.

Blue light is scattered more by the atmosphere than other colors, making it appear blue. [end of text]


### Format: Q8_0
- **Prompt**: *What is the capital of France? Answer in one word.*
- **Response**:
```text
What is the capital of France? Answer in one word. uska uska uska uskaRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEX betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory
  • Prompt: Write a short four-line poem about coding in Python.
  • Response:
    Write a short four-line poem about coding in Python.labelledbyXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessType
  • Prompt: Explain why the sky is blue in one simple sentence.
  • Response:
    Explain why the sky is blue in one simple sentence.HauptartikelHauptartikel出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年

Pipeline Logs

  • Base GGUF Conversion Peak RAM: 0.0 MB, Duration: 0.0s.
  • Logged at: Fri Jun 19 14:50:05 2026
Downloads last month
95
GGUF
Model size
9B params
Architecture
gemma2
Hardware compatibility
Log In to add your hardware

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support