Will it run on a 4090?
#1
by
brifl
- opened
Unfortunately, I had errors trying to run this on a home PC with a GeForce 4090 RTX. I was using the latest oobabooga. I'm sharing my success or failure of running these models larger than 13b locally to save some people time, disk, and bandwidth.
I didn't troubleshoot other than trying the AWQ and GPTQ versions. Both failed. The error is regarding missing config.json.