artindnr/tea-GGUF

🍵 Tea — GGUF

This repo contains GGUF quantizations of artindnr/tea, a full fine-tune of microsoft/phi-4 for question answering and long, multi-turn assistant conversations, with fine-tuning focused on Farsi (Persian) conversational ability.

These files are for use with llama.cpp and any framework built on top of it (llama-cpp-python, Ollama, LM Studio, koboldcpp, text-generation-webui, etc.).

Model Details

• Base model: artindnr/tea (full fine-tune of microsoft/phi-4, 14B parameters)
• Quantized by: artindnr
• Format: GGUF
• License: MIT
• Languages: Farsi (primary conversational focus), English, and general multilingual support

Available Files

File	Quant Type	Notes
tea.F16.gguf	F16	Full 16-bit precision, no quantization loss. Largest file, highest quality, highest VRAM/RAM requirement.
tea.Q8_0.gguf	Q8_0	Near-lossless 8-bit quantization. Best quality among the quantized options, still fairly large.
tea.Q5_K_M.gguf	Q5_K_M	Good balance of quality and size; recommended if you have the memory to spare over Q4.

As a rule of thumb: use Q5_K_M for a quality/size sweet spot, and Q8_0 or F16 when you have the VRAM/RAM to spare and want output as close as possible to the unquantized model. For the smallest, fastest option, see the separate 4-bit repo.

Downloading

# Requires: pip install -U "huggingface_hub[cli]"
huggingface-cli download artindnr/tea-gguf tea.Q5_K_M.gguf --local-dir .

Swap the filename for whichever quant you want to use.

How to Use

llama.cpp (CLI)

# Build llama.cpp if you haven't already
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON   # drop -DGGML_CUDA=ON for CPU-only builds
cmake --build build --config Release -j

# Run an interactive chat session
./build/bin/llama-cli \
  -m ./tea.Q5_K_M.gguf \
  -c 8192 \
  -n 1024 \
  --temp 0.7 \
  -cnv \
  -p "You are tea, a helpful assistant."

-cnv enables chat mode using the model's built-in chat template, so Farsi prompts and multi-turn conversation work as expected. Increase -c (context size) for longer conversations.

llama.cpp (OpenAI-compatible server)

./build/bin/llama-server \
  -m ./tea.Q5_K_M.gguf \
  -c 8192 \
  --host 0.0.0.0 \
  --port 8080

Then query it like any OpenAI-compatible endpoint:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tea",
    "messages": [
      {"role": "user", "content": "تو کی هستی و اسمت چیه؟"}
    ],
    "temperature": 0.7,
    "max_tokens": 1024
  }'

llama-cpp-python

pip install llama-cpp-python
# add CMAKE_ARGS="-DGGML_CUDA=on" before pip install for GPU acceleration

from llama_cpp import Llama

llm = Llama(
    model_path="./tea.Q5_K_M.gguf",
    n_ctx=8192,
    n_gpu_layers=-1,  # offload all layers to GPU; set 0 for CPU-only
)

response = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "تو کی هستی و اسمت چیه؟"}
    ],
    temperature=0.7,
    max_tokens=1024,
)

print(response["choices"][0]["message"]["content"])

Ollama

# Create a simple Modelfile pointing at the GGUF you downloaded
cat > Modelfile <<'EOF'
FROM ./tea.Q5_K_M.gguf
EOF

ollama create tea -f Modelfile
ollama run tea

Intended Use

Same as the base artindnr/tea model: Farsi-first conversational assistance, question answering, and long multi-turn assistant deployments — now packaged for efficient local/CPU-and-consumer-GPU inference via llama.cpp and compatible runtimes.

Limitations

• Quantization trades off some accuracy for size/speed; lower-bit quants (Q5_K_M) will show more quality degradation than Q8_0/F16, particularly on nuanced or long-context Farsi generation. For the smallest/fastest option, see the 4-bit repo, which trades off more quality for size.
• Inherits all limitations of the base artindnr/tea model and the underlying microsoft/phi-4 checkpoint, including possible hallucinated facts.
• No formal safety fine-tuning beyond what is inherited from the base model has been applied; use appropriate safeguards in production settings.

License

This model is released under the MIT License, consistent with artindnr/tea and the base microsoft/phi-4 model.

Citation

If you use tea in your work, please cite:

@misc{tea,
  title  = {tea: A Farsi-Focused, Full Fine-tune of Phi-4 for QA and Long-form Assistance},
  author = {artindnr},
  year   = {2026},
  url    = {https://huggingface.co/artindnr/tea}
}

Acknowledgements

Built on top of artindnr/tea, itself a full fine-tune of microsoft/phi-4. GGUF conversion and quantization via llama.cpp.