UNSERNSOREDapi - offline kit

Run a chat model on your own machine. After the one-time setup there is no internet, no API key and no usage limit.

What is actually in here

The kit installs Ollama and downloads Qwen 2.5 7B (qwen2.5:7b), an open-weight model published by Alibaba. Then it builds a local model called unserndored-local on top, carrying the same personality instructions the hosted UNSERNSOREDapi endpoint uses, so the answers come back in the same voice.

To be straight with you: this is not a model UNSERNSOREDapi trained from scratch. Training one needs a cluster of GPUs and a dataset far larger than a small project has. What you get here is a real chat model you own outright and run locally, with our personality layer on top of it.

Setup

chmod +x setup.sh
./setup.sh

Talk to it

python3 chat.py          # needs nothing but Python 3
node chat.mjs            # needs Node 18 or newer
ollama run unserndored-local   # or just use Ollama directly

The files

setup.sh - installs Ollama, downloads the weights, builds the persona model
Modelfile - the personality and the sampler settings
system-prompt.txt - the instructions on their own
chat.py - a terminal chat client, standard library only
chat.mjs - the same thing for Node

Serving several people at once

The two clients in here are single-user terminals. To serve a crowd, call Ollama's HTTP API from your own app instead - one request per person, each carrying that person's own message history, kept in your database under their user id. Nothing is shared between them.

Ollama answers a fixed number of requests in parallel. Raise it before you start Ollama:

OLLAMA_NUM_PARALLEL=4 ollama serve

Every parallel slot costs memory, so raise it in step with the RAM you have. Requests over the limit wait their turn instead of failing.

Reading images offline

The model in this kit answers text only. To have it read pictures on your own machine, point MODEL in setup.sh at a vision model from Ollama's library and re-run it - the persona and the rest of the kit work exactly the same way.

What you need

About 4.7 GB of disk space for the weights
8 GB of RAM is comfortable, 16 GB if you pick a 7B model
macOS, Linux, or Windows

Check the licence of Qwen 2.5 7B before shipping it inside a commercial product. Open weights are not always the same thing as open source.

Rather use the hosted endpoint?

Your API key from the dashboard reaches the same personality over the network with nothing to install. This kit is for when you want no network at all.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support