Why is this model so sloooooooow in text-generation-webui?

#6
by cleverest - opened

This is the only 30b model that is unusable due to this incredible slowdown...What do I need to do to speed this up to a usable speed? I have a 4090, 96GB of Ram, I9-13900k system.

Same here, my system us: Windows 11, RTX4090, 65GB RAM, i7 13700KF.
I have around 0.2 t/s. I wish someone could make not 128g version :(

Screenshot 2023-05-27 154224.png

Glad it's not just me! I wish I could try this out and actually use it. It's mind-numbingly slow and tries to be verbose while being that way...

Yeah, I get this (0.60) and it's mind-numbingly annoyingly unusable. It's clearly broken. No other 30b is even close to being this slow.
image.png

I guess they don't care to respond with a fix or a solution upcoming, etc...seems random and rambly anyways of a model.

Sign up or log in to comment