Delete those GGUF
Stop encouraging inferior GGUFs. GGUF format is not meant for AI image and video models, leading to degraded loss. INT8 Convrot will run on a 6GB VRAM card with ComfyUI. It is the standard now.
You should delete the GGUF models.
Agreed, these broken gguf's that do not support all features are not helpful. Besides I appreciate VRAi for the release it makes no sense to release broken checkpoint files.
GGUF is just a container, my man. The way it is dequantized depends on the inference backend.
It can run as fast as int8_convrot quant if you want. But comfyanonymous doesn't want to support it.
GGUF is just a container, my man. The way it is dequantized depends on the inference backend.
It can run as fast as int8_convrot quant if you want. But comfyanonymous doesn't want to support it.
Unfortunately, it is not "just" a container. The way it can hold the quanting of keys is different and is not the same architecture like safetensors do. It is fundamentally different and that's why the gguf lacks the features.