Spaces:

ggml-org

/

gguf-my-repo

like 520

Running on A10G

Imatrix support is here :) along with a fallback training data txt which does quite well as a default (at least for those unfamiliar they get good perplexity compared to other older options out there).
More params like -ow could fit into an advanced settings gr.accordion though being very niche and usually worse to have they're skipped for now as they're so by default.

Love it! reviewing it now @SixOpen - do you have any numbers on the overall time it takes to create the imat file and so on?

Also, the diff looks quite off, is it possible for you to update the PR to just have the diff for the code addition please! 🙏

Of course! I've ran quick tests again on some models and they were done in quite short time, for example Nous-Capybara-7B-V1.9-IQ3_XXS-GGUF step 1 was printed at around 00:01 and step 2 at 00:11 :)

And indeed the diffs are very off, it seems that as I used a roundabout way to push due to encountering this

they're like that, and since the PR is already out of draft mode I'll have to open a new one :/, will do in a bit from my other device! 😄