raymsm/Qwen2.5-7B-Instruct-GGUF

Qwen2.5-7B-Instruct GGUF

Quantized GGUF versions of Qwen/Qwen2.5-7B-Instruct generated using llama.cpp on an Oracle Cloud VM.

Repositories & Quantization Details

This repository contains the following quantization formats:

Quantization Type	File Size	Quantization Duration	Peak RAM Used	Description
Q4_K_M	4.36 GB	85.3s	18729.3 MB	Medium-sized, good balance of size/perplexity.
Q5_K_M	5.07 GB	60.8s	18729.3 MB	Medium-sized, good balance of size/perplexity.
Q6_K	5.82 GB	41.6s	18729.3 MB	Medium-sized, good balance of size/perplexity.
Q8_0	7.54 GB	20.0s	18682.9 MB	Medium-sized, good balance of size/perplexity.

Sanity Check Prompts & Responses

Here are the responses generated by the quantized models to verify coherent text generation.

Format: Q4_K_M

• Prompt: What is the capital of France? Answer in one word.
• Response:

  What is the capital of France? Answer in one word. Paris. [end of text]
  

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python. In Python, I find my muse,
  Syntax flows like gentle prose,
  A world of logic, pure and true,
  Where problems solve, like magic prose.
  

Note: I've used "prose" twice in the last line, but I can change it if you prefer. Here's an alternative version:

In Python

- **Prompt**: *Explain why the sky is blue in one simple sentence.*
- **Response**:
```text
Explain why the sky is blue in one simple sentence. The sky appears blue because the Earth's atmosphere scatters short-wavelength blue light from the sun more than other colors, a phenomenon called Rayleigh scattering. 

(Note: I've kept it to one sentence as requested, but I understand that a full explanation would require more detail.) 

Is there a specific age or grade

Format: Q5_K_M

• Prompt: What is the capital of France? Answer in one word.
• Response:

  What is the capital of France? Answer in one word. Paris. [end of text]
  

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python. Syntax and logic flow so neat, Python scripts run with ease, A language simple yet so sweet, Let's dive into the code, and seize. [end of text]
  

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

  Explain why the sky is blue in one simple sentence. The sky appears blue during the day because the Earth's atmosphere scatters short-wavelength blue light from the sun more than other colors.
  

Is that explanation too simple? If so, how can I make it more concise while still being accurate? The sky is blue because the atmosphere scatters sunlight more in the blue part

### Format: Q6_K
- **Prompt**: *What is the capital of France? Answer in one word.*
- **Response**:
```text
What is the capital of France? Answer in one word. Paris. [end of text]

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python. Syntax and logic, a programmer's delight, In Python's embrace, we write and unite, Lines of code, like verses in a rhyme, A language that's both elegant and sublime. [end of text]
  

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

  Explain why the sky is blue in one simple sentence. The sky appears blue because the Earth's atmosphere scatters short-wavelength light, like blue and violet, more than long-wavelength light, like red and orange, a phenomenon known as Rayleigh scattering.
  

To make it even simpler: The sky is blue because blue light bounces off air molecules more than other colors

### Format: Q8_0
- **Prompt**: *What is the capital of France? Answer in one word.*
- **Response**:
```text
What is the capital of France? Answer in one word.鲱方方面版权所有$MESS$MESS$MESS$MESSangkan$MESS哂哂 (~(hores volunteershores녔녔녔 volunteerhores volunteershores volunteers녔녔녔녔녔녔녔녔 volunteers녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔 MorgGLfloat volunteer MorgGLfloat volunteers녔 页面녔 页面녔녔 Morg volunteer

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python. dateFormatter isLoggedIn isAdminGLfloat泞泞泞泞GLfloat NDEBUG NDEBUG NDEBUGGLfloatGLfloatGLfloat。www个交易泞个交易}->{GLfloat.onOptionsItemSelected。www个交易}->{个交易GLfloat个交易.onOptionsItemSelected个交易}->{GLfloat.onOptionsItemSelected녔GLfloat volunt。www녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔
  

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

  Explain why the sky is blue in one simple sentence.★★PRIVATE👇}->{시스}->{녔}->{👇}->{。www녔녔녔👇}->{}->{}->{}->{}->{}->{}->{}->{}->{}->{}->{녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔녔
  

Pipeline Logs

• Base GGUF Conversion Peak RAM: 4193.3 MB, Duration: 95.7s.
• Logged at: Fri Jun 19 18:44:00 2026