Text-gen-webui said not support the 'baichuan' model type

#2
by OceanRing - opened
  1. pip install auto-gptq -> installed latest auto-gptq version 0.2.2
  2. downloaded the baichuan-7B-GPTQ to models/
  3. python server.py --trust-remote-code
    Error: auto_gptq/modeling/_utils.py line 125, in check_and_get_model_type
    TypeError(config.model_type) isn't supported yet.

Thank you so much!

update:
I find where is the issue. I need to recompile the latest auto-gptq.

Yeah, Baichuan isn't yet supported in the released AutoGPTQ. Build it from source and it will work.

AutoGPTQ 0.3.0 should be released next week and then the normal AutoGPTQ release will have Baichuan support automatically.

Sign up or log in to comment