Is imatrix possible?
#1
by mirifiuto - opened
Is there any chance I could get imatrix for this and the lm version please?
Thanks
I guess not, unfortunately. The whole thing was vibe-engineered but I might re-do it completely (Not a promise).
Walltim3 changed discussion status to closed
Is there any chance I could get imatrix for this and the lm version please?
Thanks
No imatrix — the JSON files here are REAP scores (used to pick which experts to keep during pruning), not an imatrix for quantization. You’d need to run imatrix calibration yourself on the pruned weights if you want to make good low-bit GGUFs. I believe this would be like 5 bucks in a rented computer.
Thanks.