RuntimeError: Unrecognized tensor type ID: AutocastCUDA
#2
by
nickykay
- opened
I used the code in https://huggingface.co/TheBloke/Asclepius-13B-GPTQ#you-can-then-use-the-following-code to try to load the gptq-8bit--1g-actorder_True model revision, but when trying to generate text, I get the error below in Auto-GPTQ.
I'm currently using torch 2.1.0 on an AWS G5 instance with an A10 GPU. Online sources have suggested to downgrade to torch 1.6 but that results in the same issue. How can this be resolved?