psyonic-cetacean-20B iMat GGUF

psyonic-cetacean-20B quantized using importance matrix (with love).

Uploading some of the newer quants by request, starting with IQ3.

For more information on the latest importance matrix quants and how they stack up to legacy quantization methods please check out this PR.

Need a different quantization/model? Please open a community post and I'll get back to you - thanks

GGUF

Model size

20B params

Architecture

llama

2-bit

3-bit

4-bit

Inference API

Unable to determine this model's library. Check the docs .

Collection including InferenceIllusionist/psyonic-cetacean-20B-iMat-GGUF