PollardWeights/Qwen2.5-1.5B-Instruct-Pollard

Qwen2.5-1.5B-Instruct — Pollard Weights (memory-fit GGUF)

A memory-fit GGUF build of Qwen/Qwen2.5-1.5B-Instruct, produced by Pollard Weights — models built for your machine's memory, not for a bit-width chart.

Instead of one bit-width everywhere, the sensitive tensors keep high precision and the bulk carries the compression, with the whole build sized to a target RAM budget. The output is a normal GGUF: it runs in stock llama.cpp, Ollama, or LM Studio unchanged.

Builds — pick your RAM tier

Three role-aware builds off the same f16 source. All share a Q4_K_M bulk; the sensitive tensors (token embeddings, attention q/k/v/o, norms, routers, output head) are protected at increasing precision as the budget grows. Smaller is faster; larger is higher fidelity.

file	size	protected @	for	verified on M4 (16 GB)
…-Pollard-fit1GB.gguf	834 MB	q4_K	~1 GB devices / phones	✅ 93.1 tok/s
…-Pollard-fit2GB.gguf	1.2 GB	q6_K	~2 GB (balanced)	✅ 69.8 tok/s
…-Pollard-fit3GB.gguf	1.5 GB	q8_0	~3 GB (max fidelity)	✅ 58.0 tok/s

The idea: a flat Q4_K_M treats every tensor the same. These spend extra bits where accuracy is most sensitive (attention + embeddings) and take them back on the FFN bulk — a smarter Q4, tuned to how much RAM you actually have. Builder: pollard-fit (drives llama.cpp per-tensor quantization).

Verified

Every build was loaded and generated from on an Apple M4 Mac Mini (16 GB), llama.cpp Metal, before shipping — measure first, no claim before a number. Each produced coherent, correct output (e.g. "Name three primary colors and one fact about the sky" → "…red, blue, and yellow. The sky is primarily blue due to the scattering of sunlight by the Earth's atmosphere."). Speeds are in the table above.

Run it

Pick the file for your RAM tier (example uses the balanced 2 GB build — swap the filename for fit1GB or fit3GB as you like):

# llama.cpp
llama-cli -m Qwen2.5-1.5B-Instruct-Pollard-fit2GB.gguf -p "Hello!"

# Ollama (Modelfile: FROM ./Qwen2.5-1.5B-Instruct-Pollard-fit2GB.gguf)
ollama create qwen2.5-1.5b-pollard -f Modelfile && ollama run qwen2.5-1.5b-pollard

Or point LM Studio at the file — it's a standard GGUF.

Reproduce it

git clone https://github.com/WestWaters/pollard-weights && cd pollard-weights
./install.sh
# fit the f16 source to your RAM budget — the number picks the tier:
pollard-fit --gguf Qwen2.5-1.5B-Instruct-f16.gguf --ram 2 --reserve 0.5 \
            --out Qwen2.5-1.5B-Instruct-Pollard-fit2GB.gguf
# ladder: --ram 1.2 -> fit1GB (q4_K) · --ram 2 -> fit2GB (q6_K) · --ram 2.6 -> fit3GB (q8_0)

pollard-fit --plan-only prints the full per-tensor allocation and the exact llama-quantize command before building anything.

Check your own hardware first

pollard-calc --model Qwen/Qwen2.5-1.5B-Instruct --ram 2   # what CAN this machine do

Credits & license

• Base model: Qwen2.5-1.5B-Instruct by the Qwen team, under Apache-2.0. This build inherits that license.
• Quantization runtime: llama.cpp (ggml-org).
• Method & builder: Pollard Weights — role- and depth-aware, memory-fit GGUF builds. Apache-2.0.

Doctrine: measure first, no claim before a number.