Spaces:
Running
on
T4
This Space with other models
Do you plan doing this space with your other models or just this one?
I do have a CPU only space with a very small model here on my personal account : https://huggingface.co/spaces/Henk717/Koboldcpp-AI-Dungeon-2-Classic
This one is powered by a HF grant, its not affordable for us to provide this space for other models of proper sizes.
Instead, I tried building the space in such a way that it is incredibly easy for anyone to make their own.
Any model creator or space creator wanting to use our software for their own model spaces use the duplicate feature within Huggingface and then replace the GGUF link with the model of your choice (Make sure its the direct download link for the model). In the case you want to use the model without the GPU you need to remove the additional parameters. Your cloned space will now automatically install with the model of choice.
If you do want more model freedom you can also check out the KoboldAI Lite space, which runs on the GPU's of volunteers having a wider model selection.
There are rumors of HF allowing free T4's to a wider selection of spaces, if they end up doing this I gladly make new versions for models this space is compatible with (OPT never got ported to Llamacpp).
Thanks. I'll check options you mentioned.