Can this model work on cpu?

#21
by wiseluke - opened

"Has anyone tried running this on CPU before? I am interested in giving it a try."

does for me. EDIT: GGUF version does. Not tried it native. - EDIT 2: A bit slower than GGUF on same hardware, but ya, it worked.

I wrote a simple wrapper CLI if anyone wants to try the GGUF version on CPU :)

You can work native as well it's just a model supported by transformers

Sign up or log in to comment