Hoping this work out well!
#2
by
Olafangensan
- opened
I was so annoyed that there wasn't an in-between model for llama 3 family. 8b is great, but it's no 70b.
Having a 42b means that anyone with a 3090/4090 might actually get to play with a quantized SOTA model.
Bless you, sir!
Me too! I have a 3090 locally and that VRAM size has been increasingly neglected lately. Hopefully it works out.
Thanks for the support!
Very interesting! Any plans on doing the same with instruct version?
Wow, really interesting!
This gives some hope of running the llama-405b monster!