Ollama version please.
Please consider adding this model to Ollama.
I really enjoy interacting with the model with the llama.Cpp.
If you need an API that can run this model well KoboldCpp is also an option.
I know that i use it already.
Ollama can run GGUF files on Open WebUI frontend. I have converted most of my gguf models to use in Open WebUI. You can either download from Hugging Face or import a local GGUF that you already downloaded. It convert in minutes. I see no difference in performance nor personality of the model. Same thing as running on LM Studio. It is a graphical interface that looks like Chatgpt, has TTS, models can web search, there's a ton of incredible features. I now talk with my GGUF models hands free, it supports a microphone and voice to text too. And it accepts OpenAI API, and several others. All on a beautiful GUI. See the immense list of features. It's the greatest project I've seen on a long run. The best thing I found so far !
Open WEB UI is also OpenAI API compatible like you said so it can be paired with GGUF backends other than ollama.
I converted the F16 model to an Q4_K_M for Olama, but I want to know if I have the permission to upload it to the Olama repository on their site ?
Yes, go ahead. Just make sure you label it properly. This is v2a btw, a test model.
Open WEB UI is also OpenAI API compatible like you said so it can be paired with GGUF backends other than ollama.
Exactly. I didn't test yet, but as I use LM Studio for gguf models, I could pair Open WebUI with LMStudio, and use all it's features, like TTS hands free, RAG, Web Search, Long term memory, image generation, filters, tools, download, fine tune models, and convert other kinds of models. I'll do some tests. 👍
Yes, go ahead. Just make sure you label it properly. This is v2a btw, a test model.
Thank you.
When I try to push it to Ollama, I keep getting "Retrieving manifest | Error: file does not exist." I keep investigating.
May be a test model but it's so much fun cheating with it.