KoboldAI

/

OPT-13B-Nerybus-Mix

like 32

@TheBloke ? Do you think you could make a GGML version of this, please? :) I think it would fit in 16GB of RAM, right? Well, I believe these 13B models do, even though they tend to be a bit slower than 7B. :)

OPT has no GGML port. I would have converted these officiallly if it had. So there isn't anything that can be done to convert them until someone makes an OPT implementation for GGML.