Will it run on a 4090?
#1
by
brifl
- opened
Hell yes it will. I ran this successfully on a home PC with a GeForce 4090 RTX. It ran on oobabooga. I'm sharing my success or failure of running these models larger than 13b locally to save some people time, disk, and bandwidth.
This is the best code generator I've used so far. It's amazing how far this has come in a year.