TheBloke

/

Nous-Hermes-13B-GPTQ

like 172

I am getting the error:
FileNotFoundError: Could not find model in TheBloke/Nous-Hermes-13B-GPTQ

Am I missing something in my code below (running on collab)?:

model_name_or_path = "TheBloke/Nous-Hermes-13B-GPTQ"
model_basename = "nous-hermes-13b-GPTQ-4bit-128g.no-act.order"

use_triton = False

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)

model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
        model_basename=model_basename,
        use_safetensors=True,
        trust_remote_code=True,
        device="cuda:0",
        use_triton=use_triton,
        quantize_config=None)