Smaller undermerger?
The undermerger file is the only thing keeping me from utilizing the model, is theee a way to quant it or gguf it to a smaller size?
Q8 ，will upload soon.
i appreciate the transformer gguf!
i was speaking about the undermerger file though, the "JoyAI-Image-Und-merger_bf16.safetensors"
that file seems to take a lottttttt of vram usage. :(
OK,maybe Q8? or Q6?
OK,maybe Q8? or Q6?
either Q would probably work, if you dont mind me asking... what are you using to quant them down? llama.cpp?
ive tried in the past to do it but i never get it right
Hey brother, just checking in to see if youve made any progress with the under merger quanting
upload now
Your awesome!
so slowly ，maybe need 2 hours...
so slowly ，maybe need 2 hours...
i am happy to wait! thank you very much!
hey so i tested the Q6 Transformer and got a bunch of errors, i corrected the node errors but the issue is now that im just getting really noisey outputs. i suspect that the Q6 gguf had some sort of compression problem or that its simply stripped down too far for the decoding process. heres a few of my test generations from the Q6. im testing the Q8 now to see if that fixes the problem.
original image for reference
tests from Q6
the first 3 test images were before i corrected the vae issue but it still persists even though the math is correct. i think that if the Q8 works, its simply that the Q6 is too striped down
the Q8 also is giving the same bad outputs. ive done all i can on my end as far as recoding the nodes and ensuring everything is mathmatically correct. i believe the quantization process was incorrect
Your transformer verion ?
It looks like transformer>4.58.0 cause a bad resulet,
this gguf don't support gguf cpp or kijai 's node.
my test use your photo,
D:\AI_Tools\ComfyUI_windows_portable>python_embeded\python.exe -c "import transformers; print(transformers.version)"
4.57.6
im not sure whats wrong, should i downgrade my transformer further down?
or maybe diffuser verson ，try
diffusers>=0.37.0.dev0
this node use diffusers codes to deal with gguf
windows_portable need
python -m to install it
D:\AI_Tools\ComfyUI_windows_portable>.\python_embeded\python.exe -m pip show diffusers
Name: diffusers
Version: 0.37.1
from now on, ill just leave the coding to you brother your the KING of this!
my nodes obviously were TERRIBLE but yours work PERFECTLY! <3 ill be sure to give you full credit in my Youtube video today! expect some downloads from my community!
