Hugging Face's logo

huihui-ai
/
Huihui-gemma-4-12B-it-qat-q4_0-unquantized-abliterated

Any-to-Any
Transformers
Safetensors
gemma4_unified
image-text-to-text
abliterated
uncensored
Model card Files
xet
 Community
2

would there be one for 25b-a4b ?

#2
by patientxtr - opened
Discussion
patientxtr

or other QAT gemma from google ?
nonetrix

There is QAT for every version, I can only assume they are working on uncensoring the others, really excited to get my hands on 31B version
huihui-ai
 Owner

Yes, please wait patiently.
Jahaz

Also can u make Stepfun flash 3.7 pls?
huihui-ai
 Owner

Also can u make Stepfun flash 3.7 pls?
Will give it a try later.
patientxtr

thanks for the 26b !
patientxtr changed discussion status to closed
nonetrix

I'm able to do 49K context on my 16 GB card with the official Gemma QAT 31B weights on Ollama, but with the uncensored QAT weights I had to lower it or get out of memory errors weirdly, might be Ollama's fault partially for not calculating memory usage very well and just offloading correctly but it seems the model is slightly larger than Google's QAT version.

Also, my main use-case is creative writing, and maybe just me, but it feels a bit different than normal QAT Gemma much more of slop feeling. Overall though, model is nice, and at least feels smarter than normal old Q4

Sign up or log in to comment