luispoveda93/MiniCPM5-2B-catalan-chat-GGUF

MiniCPM5-2B-catalan-chat — GGUF

GGUF conversions of luispoveda93/MiniCPM5-2B-catalan-chat — a LoRA fine-tune of openbmb/MiniCPM5-2B on projecte-aina/InstruCAT (165,100 Catalan instructions, 1 epoch) for conversational Catalan.

Converted with llama.cpp convert_hf_to_gguf.py (fp16) and quantized with llama-quantize. The chat template is embedded in the GGUF metadata (tokenizer.chat_template).

Available quants

File	Quant	Size	Notes
MiniCPM5-2B-catalan-chat-Q4_K_M.gguf	Q4_K_M	1.6 GB	Best size/quality trade-off for most uses
MiniCPM5-2B-catalan-chat-Q8_0.gguf	Q8_0	2.7 GB	Near-lossless
MiniCPM5-2B-catalan-chat-f16.gguf	F16	5.0 GB	Reference / re-quantization source

Usage

llama.cpp

llama-cli -m MiniCPM5-2B-catalan-chat-Q4_K_M.gguf \
  --chat-template llama3 \
  -p "Ets un assistent conversacional que respon sempre en català.\nUser: Hola! Com estàs?"

Or with the server (the embedded chat template is applied automatically):

llama-server -m MiniCPM5-2B-catalan-chat-Q4_K_M.gguf --port 8080

Ollama

Create a Modelfile:

FROM MiniCPM5-2B-catalan-chat-Q4_K_M.gguf
SYSTEM "Ets un assistent conversacional que respon sempre en català."

ollama create minicpm5-catalan -f Modelfile
ollama run minicpm5-catalan

Details

• Base model: openbmb/MiniCPM5-2B (2.5B params, Llama architecture, vocab 130,560)
• Training: LoRA r=32/α=64, lr 2e-4, effective batch 32, max_length 2048 packed, completion-only loss; final loss ≈ 0.33, token accuracy ≈ 0.92 (see training metrics)
• License: base model Apache-2.0; training data CC-BY-NC-ND-4.0 (non-commercial) — check InstruCAT terms before commercial use
• Limitations: single-turn task-oriented training data; general chit-chat behavior comes from the base model