Spaces:

ggml-org

/

gguf-my-repo

like 902

Running on A10G

Maybe someone can give me an answer or a link because I can't find anything about it.

@mishig can share an awesome link he created recently

Thanks. This is what I needed:

https://www.reddit.com/r/LocalLLaMA/comments/1ba55rj/overview_of_gguf_quantization_methods/?rdt=36175

@multiverse we created this doc page https://huggingface.co/docs/hub/gguf#quantization-types

@julien-c @mishig Maybe we can also see the labelled file type in the GGUF visualizer: "general.file_type"->12->Q3_K_M.
Example here phymbert/dbrx-16x12b-instruct-q3_k_m-gguf

^yes 👍

@mishig had you seen this reddit page? https://www.reddit.com/r/LocalLLaMA/comments/1ba55rj/overview_of_gguf_quantization_methods/?rdt=36175 looks quite awesome (thanks @multiverse for sharing)

Like choosing which sandpaper to use. :)