TheBloke

/

Mistral-7B-OpenOrca-AWQ

like 40

When I attempt to load an awq model it fails with error:

ImportError: DLL load failed while importing awq_inference_engine: The specified module could not be found.

i've tried with pip install and conda install but there is no package named awq_inference_engine.

I'm using the latest textgen-webui on windows, installed with one click installer, cuda 12.1 on 4070Ti.

pip install awq returns requirements already satisfied for everything, as expected.

I have also pip installed llm-awq in the same conda env as this is all that popped up when i googled awq_inference_engine.

What am I missing? Forgive my noobiness.

Thanks in advance...