NoRobots is not in the training data for this one?
I found Lumimaid-v0.2-8B-Q6_K_L-imat.gguf to load with KoboldCPP. Is this official?
I found Lumimaid-v0.2-8B-Q6_K_L-imat.gguf to load with KoboldCPP. Is this official?
Hello, we will do our official quant as soon as this PR : https://github.com/ggerganov/llama.cpp/pull/8676 will be merged with main.
The other quant aren't official, and I can't tell you if they work or not with the version of koboldcpp you have.
I found Lumimaid-v0.2-8B-Q6_K_L-imat.gguf to load with KoboldCPP. Is this official?
Hello, we will do our official quant as soon as this PR : https://github.com/ggerganov/llama.cpp/pull/8676 will be merged with main.
The other quant aren't official, and I can't tell you if they work or not with the version of koboldcpp you have.
Thank you! I'll wait!
All llama-3.1 quants done with any released llama.cpp version will have wrong rope setups (and might stop working once a fix is done). They should work if you manually specify --contextsize 8192 with current koboldcpp, but it's best to switch to new quants (or even official ones :) once available.
All llama-3.1 quants done with any released llama.cpp version will have wrong rope setups (and might stop working once a fix is done). They should work if you manually specify --contextsize 8192 with current koboldcpp, but it's best to switch to new quants (or even official ones :) once available.
Heyy welcome you, thx for your work btw
We will probably do some quant and let the peeps do the others haha, we burned our compute ressource
Sigh. I'll be busy for weeks with llama-405b alone. And then the other 50 llama-3.1 based models in the queue. But the new lumimaid variants are at the top of the list, so there is no break for anybody :)