Which ones work with GPU inference? I haven't tested all of them but I have a feeling that only the basic Q4 does. It would be a waste having to DL all of them for the users seeking hardware-accelerated inference.
· Sign up or log in to comment
