mosaicml

/

mpt-7b

like 1.14k

Yes, you can run on CPU with the standard model.generate() instructions as long as you have enough RAM to hold the model weights. This is ~30GB if using FP32 or ~15GB if using BF16.

Yes, you can run on CPU with the standard model.generate() instructions as long as you have enough RAM to hold the model weights. This is ~30GB if using FP32 or ~15GB if using BF16.

Thanks, Also I’m having trouble to load the model using oobabooga webui. Any instructions how can I use this model with the WebUI framework?

We leave it to the developers of oobabooga to add support for our model architecture.