raymsm/Phi-4-GGUF

Phi-4 GGUF

Quantized GGUF versions of microsoft/Phi-4 generated using llama.cpp on an Oracle Cloud VM.

Repositories & Quantization Details

This repository contains the following quantization formats:

Quantization Type	File Size	Quantization Duration	Peak RAM Used	Description
Q4_K_M	8.44 GB	145.9s	31911.6 MB	Medium-sized, good balance of size/perplexity.
Q5_K_M	9.78 GB	119.4s	31911.6 MB	Medium-sized, good balance of size/perplexity.
Q6_K	11.20 GB	146.1s	31911.6 MB	Medium-sized, good balance of size/perplexity.
Q8_0	14.51 GB	120.2s	31911.3 MB	Medium-sized, good balance of size/perplexity.

Sanity Check Prompts & Responses

Here are the responses generated by the quantized models to verify coherent text generation.

Format: Q4_K_M

• Prompt: What is the capital of France? Answer in one word.
• Response:

  What is the capital of France? Answer in one word. [end of text]
  

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python. The poem should rhyme and mention the concepts of 'variables', 'loops', and 'functions'. [end of text]
  

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

  Explain why the sky is blue in one simple sentence. [end of text]
  

Format: Q5_K_M

• Prompt: What is the capital of France? Answer in one word.
• Response:

  What is the capital of France? Answer in one word. [end of text]
  

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python. [end of text]
  

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

  Explain why the sky is blue in one simple sentence. [end of text]
  

Format: Q6_K

• Prompt: What is the capital of France? Answer in one word.
• Response:

  What is the capital of France? Answer in one word. [end of text]
  

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python. The poem must rhyme and include the words "syntax" and "loop."
  

Answer

In Python's realm where the syntax is neat, A loop spins round, a rhythmic feat. With each command, the code does swoop, In this dance of logic, we find our loop. [end of text]

- **Prompt**: *Explain why the sky is blue in one simple sentence.*
- **Response**:
```text
Explain why the sky is blue in one simple sentence. [end of text]

Format: Q8_0

• Prompt: What is the capital of France? Answer in one word.
• Response:

  What is the capital of France? Answer in one word. FAILURE strictly strictlyibilibil Pir Sanfordbiratern Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur Bur(exec Bur Bur Bur Bur Bur Bur Bur
  

• Prompt: Write a short four-line poem about coding in Python.
• Response:

  Write a short four-line poem about coding in Python.nymnymnymnymnym-docnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnymnym Grosnym^K^K Roths Grosnym^K Gros^KCelekö^K Roths Roths Roths径 Roths '\" Roths Roths Roths Roths '\" Roths Roths Roths Roths Roths^K Roths Roths Roths
  

• Prompt: Explain why the sky is blue in one simple sentence.
• Response:

  Explain why the sky is blue in one simple sentence.Teacher " CwikAimage-in-inur-forururur-forurururururvaulturururvaultvaultvaultvault_codegenvault239239239239239239239239239239239 gh239239239239239urd239
  

Pipeline Logs

• Base GGUF Conversion Peak RAM: 4080.6 MB, Duration: 294.2s.
• Logged at: Fri Jun 19 18:11:27 2026