Experimental imatrix quantizations, at 16K context, using a cocktail of data taken from the exllamav2 repo, kalomaze's random token tests and some stories.

Consider them experimental! But they may be better than the equivalent non-matrix GGUFs. Or they may not!

Downloads last month: 332

GGUF

Model size

34.4B params

Architecture

llama

Unable to determine this model’s pipeline type. Check the docs .

Invalid base_model specified in model card metadata. Needs to be a model id from hf.co/models.

Collection including brucethemoose/Yi-34B-200K-RPMerge-iMat.GGUF

Most Recent Merge

Collection

7 items • Updated Feb 10 • 1