fl2va_pruned_fp8_Q8_0.gguf and fl2va_pruned_fp8_Q8_CR.gguf are identical files (same hash)
Hi molbal, thanks for providing these quants!
I noticed that minimax_h3_fl2va_pruned_fp8_Q8_0.gguf and minimax_h3_fl2va_pruned_fp8_Q8_CR.gguf have the same file size (20.2 GB) and identical hashes — it looks like the Q8_CR file may have been accidentally uploaded twice, once under the Q8_0 filename.
I downloaded the "Q8_0" file expecting a standard GGUF quant, but when loading it in ComfyUI the log shows:
gguf qtypes: F32 (532), I8 (264)
Detected mixed precision quantization
The INT8 tensors then route through comfy_kitchen's int8_linear → torch._int_mm, which is not implemented on Apple Silicon (MPS), so it crashes with:
NotImplementedError: The operator 'aten::_int_mm' is not currently implemented for the MPS device
For comparison, the true Q8_0 of Ref2VA (21.4 GB) loads fine as a standard quant.
Would it be possible to re-upload the actual standard Q8_0 for the FL2VA model? That would make it usable on Mac/MPS as well. Thanks!
Oh I'm sorry I probably mixed it up. I will run the quant script again,and reupload it!
No worries, thanks!
I uploaded it again, this time hopefully the right one 😃
https://huggingface.co/molbal/MiniMax-H3-GGUF/commit/b45874371c61c49bf04096602f5527aac71b360b
I tested the updated one, and it works fine. Thanks!
Glad to hear!