Any chances for higher bpw release?
Like "RP-Stew-v2.5-34B-exl2-5"
Like "RP-Stew-v2.5-34B-exl2-6"
Like "RP-Stew-v2.5-34B-exl2-8"
?
Would that significantly increase quality? Mm... probably it is a hard question to answer.
From what I'm told and seen with graphs, the quality increase starts to have diminishing returns past 4.65 when comparing V/RAM usage to performance. It's still better past that, don't get me wrong, and if you can run a bigger quant you should, but the jump from say 4.65>6.0 isn't as obvious compared to even 3.5>4.0
As for bigger quants from myself, hmm... not at this moment. I prefer testing them first before putting them out into the wild, since on some occasions they can turn out bad when being baked, and I don't have a rig to really test anything at this size beyond 5.0