Quant request: https://huggingface.co/CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512
Please quantize https://huggingface.co/CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512
Polish 70B instruct model from Ministry of Digital Affairs of Poland (HIVE AI consortium).
Based on Llama 3.1, license: llama3.1.
Preferred quants: Q4_K_M, Q5_K_M
Hi! I can definitely do that for you if still needed. I'm currently running a training job, but as soon as it finishes this evening, I'll start working on the Q4_K_M and Q5_K_M quants. I'll drop the link here once they are ready!
@adzak
🇵🇱 Llama-PLLuM-70B-instruct-2512 — GGUF
GGUF quantizations of CYFRAGOVPL/Llama-PLLuM-70B-instruct-2512 — the largest Polish instruction-tuned LLM by the PLLuM / HIVE AI consortium.
Quantized by @Mati83moni using llama.cpp (build 465b1f0) on AWS EC2.
⚠️ Important note about quantization resources:
The reported ~3 GB refers to processing overhead, not model size.
Memory requirements depend on runtime, context length, and offload settings.
[Mati83moni/Llama-PLLuM-70B-instruct-2512-GGUF]
(https://huggingface.co/Mati83moni/Llama-PLLuM-70B-instruct-2512-GGUF)
ENJOY 😁 @adzak