missing quantize_config.json file

by aromo17 - opened Apr 11

Apr 11

Hello,

I am trying to evaluate the model in lm-harness. However when I tried to do it i get following error.
FileNotFoundError: [Errno 2] No such file or directory: './Qwen1.5-72B-Chat-GPTQ-Int4/quantize_config.json

I think I would need the quantize_config.json file

jklj077

Qwen org Apr 18

I think you have set the autogptq argument, which will load the model using AutoGPTQForCausalLM while this model should be loaded using AutoModelForCausalLM. Please first try removing the related arguments.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment