Error trying out Demo Code

#1
by EricBLivingston - opened

I tried out the autoawq demo code as written in the description, and got the following error:

Traceback (most recent call last):
File "C:\Users\eric\Documents\Python\ml_test\awq_test.py", line 7, in
model = AutoAWQForCausalLM.from_quantized(model_name_or_path, fuse_layers=True,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\eric\miniconda3\envs\ml_test\Lib\site-packages\awq\models\auto.py", line 49, in from_quantized
return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\eric\miniconda3\envs\ml_test\Lib\site-packages\awq\models\base.py", line 162, in from_quantized
load_checkpoint_in_model(
File "C:\Users\eric\miniconda3\envs\ml_test\Lib\site-packages\accelerate\utils\modeling.py", line 1063, in load_checkpoint_in_model
raise ValueError(f"{checkpoint} is not a folder containing a .index.json file.")
ValueError: C:\Users\eric.cache\huggingface\hub\models--TheBloke--Guanaco-13B-Uncensored-AWQ\snapshots\01a5af5abeb8cd68a1ce77a39aa5a7d28bedcd6a is not a folder containing a .index.json file.

Process finished with exit code 1

I checked and the model folder where it's looking has several json files in it but no .index.json file.

I tried again on a fresh install but I get the same error as above. How do I create or get the .index.json file it's looking for?

Sign up or log in to comment