Inference Speed comparison
#1
by zimtzi - opened
I wonder what the Inference Speed compared to Pytorch with pytorch-cross-attention is..
At the moment i have 10s / step with original model & bf16 in ComfyUI (M4 Pro 48gb) - is it worth to switch?
The bf16 speed is probably about the same, maybe slightly faster. the q8 would likely save you some time. I honestly can't tell you how this would work on pytorch. All of these models were converted to support SceneWorks which doesn't use pytorch or even python at all. It's RUST and pure MLX metal.
https://github.com/SceneWorks/SceneWorks
zimtzi changed discussion status to closed