Context?
#1
by
brucethemoose
- opened
How much of that context can you fit in 24GB with 4.65bpw?
Seems like 3-4bpw might be more of a sweetspot?
To answer my own question, its about ~27k max for 4.65bpw and ~47k max for 4bpw on a empty 3090
Thanks for the info. The other bpw models were meant to be uploaded as well, but had a misconfiguration. The usual 3, 4, 5, 6 and 8bpw versions should start popping up in an hour or so.