Instructions to use ohmysimo/Wren-512-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use ohmysimo/Wren-512-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ohmysimo/Wren-512-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf ohmysimo/Wren-512-GGUF:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ohmysimo/Wren-512-GGUF:Q8_0
# Run inference directly in the terminal:
llama cli -hf ohmysimo/Wren-512-GGUF:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ohmysimo/Wren-512-GGUF:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf ohmysimo/Wren-512-GGUF:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ohmysimo/Wren-512-GGUF:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ohmysimo/Wren-512-GGUF:Q8_0

Use Docker

docker model run hf.co/ohmysimo/Wren-512-GGUF:Q8_0

LM Studio
Jan
Ollama
How to use ohmysimo/Wren-512-GGUF with Ollama:
```
ollama run hf.co/ohmysimo/Wren-512-GGUF:Q8_0
```
Unsloth Desktop

How to use ohmysimo/Wren-512-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ohmysimo/Wren-512-GGUF:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ohmysimo/Wren-512-GGUF:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ohmysimo/Wren-512-GGUF with Docker Model Runner:
```
docker model run hf.co/ohmysimo/Wren-512-GGUF:Q8_0
```

Lemonade

How to use ohmysimo/Wren-512-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ohmysimo/Wren-512-GGUF:Q8_0

Run and chat with the model

lemonade run user.Wren-512-GGUF-Q8_0

List all available models

lemonade list

Hermes Agent

How to use ohmysimo/Wren-512-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ohmysimo/Wren-512-GGUF:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ohmysimo/Wren-512-GGUF:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ohmysimo/Wren-512-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ohmysimo/Wren-512-GGUF:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ohmysimo/Wren-512-GGUF:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Wren-512 - GGUF

Wren-512 is a mixed-precision GGUF of Swift 1.5 Qwen3.8-Flash-Next by UkisAI (itself a derivative of Qwen3.8-Flash-Next) that keeps all 512 experts instead of pruning half of them like Wren. "Swift" and "UkisAI" are trademarks of UkisAI, used here only to describe the model's origin.

Routed experts in two precision groups per layer (IQ3_XXS / IQ2_XXS gate-up, IQ3_XXS down), allocated from the measured per-expert quantisation error on a multidisciplinary quiz; attention, shared experts and the rest in Q8_0; n-gram table in Q4_0.
Healed: routers, norms and rank-16 LoRA (merged) distilled from Swift's own top-20 logits, 70 % math. Training: 4010052 tokens, best validation KL 0.0727521005954424.
Training-data note: 718 of the 10,423 healing documents are translation pairs whose reference answer was written by Gemini; besides the distillation on Swift's logits, the healing applied a cross-entropy (weight 0.5) on those references. The same holds for the Wren_T3-v2 healing.
Wren-512-MTP-q8_0.gguf: Swift's own multi-token-prediction layer (unchanged weights, Q8_0) for speculative decoding.
Needs the tiered-experts llama.cpp fork: https://github.com/OhMySimo/llama.cpp (branch tiered-experts).

Results

Quiz: 300 multiple-choice questions (6 domains x 50; ±2-3 points of noise). KLD: 25 x 2048 tokens of Swift's own held-out answers, against Swift Q8_0.

Model	Size GB	all	math	code	logic	science	commonsense	philosophy	KLD vs Swift Q8	same top token
Swift 1.5 (Q8_0, reference)	188.2	89.0	82.0	84.0	92.0	94.0	96.0	86.0	–	–%
Wren-512 B-heal (this repo)	82.3	87.0	74.0	84.0	90.0	92.0	96.0	86.0	0.081325	91.359%
Wren-512 B (before healing)	82.3	86.7	72.0	82.0	90.0	90.0	98.0	88.0	0.077427	91.82%
Wren-512 A (not published)	80.3	86.3	72.0	82.0	90.0	90.0	98.0	86.0	0.085759	91.128%
Wren_T3-v2 (pruned 50 %)	60.4	82.3	66.0	78.0	92.0	84.0	92.0	82.0	–	–%

Confabulation on API-existence questions (4000 yes/no probes, first-token probabilities, no tools): a "confident error" is a wrong answer given with probability >= 0.75.

Model	accuracy	confident errors	AUROC confidence→right
Wren_T3-v2 (pruned 50 %)	0.624	0.208	0.642
Swift 1.5 (Q8_0, reference)	0.727	0.146	0.725
Wren-512 B-heal (this repo)	0.717	0.167	0.705
Wren-512 B (before healing)	0.726	0.172	0.696
Wren-512 A (not published)	0.735	0.170	0.701

Speed (one RTX PRO 6000, model on GPU, n-gram table in RAM, greedy, 256-300 tokens):

Prompt	without MTP	with MTP draft	drafts accepted
code	79.6 t/s	168.7 t/s	97 %
step-by-step arithmetic	-	154.8 t/s	89 %
Italian prose	-	111.0 t/s	57 %

Use

# fork: git clone -b tiered-experts https://github.com/OhMySimo/llama.cpp && build with CUDA or ROCm
llama-server -m Wren-512_B-heal.gguf -ngl 99 -ot "exps=CPU,per_layer_token_embd=CPU" -fa on -c 32768 --jinja
# + MTP: -md Wren-512-MTP-q8_0.gguf --spec-type draft-mtp -ngld 99

Experts in RAM: about 45 GB of RAM for the experts plus the GPU part; the 29 GB n-gram table is read on demand. On NVIDIA set LLAMA_NO_LAYER_MK=1 (the fork's layer megakernel is tuned for AMD and does not support multi-GPU).

Files in training/: allocation (tiers/types), healing log and map, evaluation logs.