Is imatrix possible?

#1
by mirifiuto - opened

Is there any chance I could get imatrix for this and the lm version please?
Thanks

I guess not, unfortunately. The whole thing was vibe-engineered but I might re-do it completely (Not a promise).

Walltim3 changed discussion status to closed

Is there any chance I could get imatrix for this and the lm version please?
Thanks

No imatrix — the JSON files here are REAP scores (used to pick which experts to keep during pruning), not an imatrix for quantization. You’d need to run imatrix calibration yourself on the pruned weights if you want to make good low-bit GGUFs. I believe this would be like 5 bucks in a rented computer.

Thanks.

Sign up or log in to comment