how do you load it?
#1
by
KnutJaegersberg
- opened
I'd guess this might be a special bnb quant. I tried loading it it didnt work.
ValueError: Unknown quantization type, got higgs - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq', 'aqlm', 'quanto', 'eetq', 'hqq', 'compressed-tensors', 'fbgemm_fp8', 'torchao', 'bitnet']
I see, there is a pull request
so now I'm stuck with
RuntimeError: Failed to import transformers.integrations.higgs because of the following error (look up to see its traceback):
module 'pynvml' has no attribute 'nvmlDeviceGetCudaComputeCapability'
Looks like a failure with torch.cuda.get_device_capability(...). Which PyTorch and CUDA are you using?