Not supported for Apple silicon
Please release a version that is supported on Apple silicon since INT8 and FP8 both fail due to MPS backend not having certain operators implemented and the CPU fallback is extremely slow.
Keep up the good work!
BF16 works. You need 96gb unified memory.
BF16 works. You need 96gb unified memory.
Thank you, I was already aware of it but I have 64GB of memory.
need install use this https://github.com/pawel-mazurkiewicz/ComfyUI-AppleSilicon-FP8 my case successfully with M4MAX 64GB using pruned_int8_convrot.safetensors
On my M3 Max (32GB), I got a VAE error as soon as I ran it.
Could you tell me which model and workflow you used?
@jefflin > need install use this AppleSilicon-FP8 my case successfully with M4MAX 64GB using pruned_int8_convrot.safetensors
@Rhystic1 > My error was "No backend can handle 'dequantize_int8_embedding': eager: q: dtype torch.bfloat16 not in {torch.int8}"
This was a ComfyUI bug yesterday, now fixed
You can 1- re-install Comfy, or 2- replace the suggested lines in ops.py in "~/ComfyUI-Installs/ComfyUI/ComfyUI/Comfy", or 3- use this fixed ops.py file: https://jumpshare.com/s/vOhaXPnRoIOOyRjGAkYu
The default T2V workflow gave me 200s/it to 500s/it for a 7 second duration at 0.2 megapixels (608x352). PAINFUL, but more testing improved things...
CURRENT TEST RESULTS & SUMMARY
- AppleSilicon-FP8 extension is essential.
- Comfy's "pruned int8_convrot" (21GB) is 22% faster than lilcheaty's "pruned_nvfp4"(12GB), despite being twice as large.
- Spectrum adds an additional 18-20% speedup (with minor detail changes).
- Combined (int8_convrot & spectrum) completed a real world test in 24 minutes VS 48 minutes (no spectrum & nvfp4).
Next steps are trying:
- Other Quants - https://huggingface.co/Abiray/MiniMax-H3-GGUF;
- An INT4 variant - https://huggingface.co/tsolful/Minimax_H3_INT4MixedConvRot.
Will report back if I find anything else useful. Mac users really need the speedup Loras that some are working on.