Instructions to use SciTools/OnBoard with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SciTools/OnBoard with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SciTools/OnBoard:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf SciTools/OnBoard:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SciTools/OnBoard:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf SciTools/OnBoard:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SciTools/OnBoard:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf SciTools/OnBoard:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SciTools/OnBoard:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SciTools/OnBoard:Q4_K_M

Use Docker

docker model run hf.co/SciTools/OnBoard:Q4_K_M

LM Studio
Jan
Ollama
How to use SciTools/OnBoard with Ollama:
```
ollama run hf.co/SciTools/OnBoard:Q4_K_M
```

Unsloth Studio

How to use SciTools/OnBoard with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SciTools/OnBoard to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SciTools/OnBoard to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for SciTools/OnBoard to start chatting

How to use SciTools/OnBoard with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SciTools/OnBoard:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SciTools/OnBoard:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use SciTools/OnBoard with Docker Model Runner:
```
docker model run hf.co/SciTools/OnBoard:Q4_K_M
```

Lemonade

How to use SciTools/OnBoard with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SciTools/OnBoard:Q4_K_M

Run and chat with the model

lemonade run user.OnBoard-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use SciTools/OnBoard with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SciTools/OnBoard:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SciTools/OnBoard:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use SciTools/OnBoard with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SciTools/OnBoard:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SciTools/OnBoard:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

OnBoard recommended models (GGUF)

The models OnBoard's installer offers, mirrored here for stable hosting. All files are 4-bit GGUF quantizations for llama.cpp-based serving (shipped with OnBoard/Understand as ullama), all under the Apache-2.0 license, mirrored unmodified from the source repos below.

Every model here has been through our own qualification testing on real Understand project data: a chat test that measures how well the model answers questions about a codebase using OnBoard's analysis tools, and a code-summary benchmark that grades generated overviews for accuracy against the source. The notes below come from those measurements, not from vendor claims. That said, AI is AI — every model gets things wrong, so treat any answer as a starting point and take it with a grain of salt.

Which model should I use?

Run ullama-recommend (shipped with OnBoard and Understand) — it looks at your machine's memory and suggests the right size. In short: pick the largest one your machine runs comfortably, and prefer the gemma models when you want the best chat answers.

Model	Download	Why you would pick it
gpt-oss-120b	63 GB (2 parts)	The most accurate code summaries in our testing, and quick for its size. Needs roughly a 128 GB machine.
Qwen3.6-35B-A3B	22.4 GB	Highly accurate code summaries, and much faster than other models this large.
Gemma 4 12B	7.1 GB	The best chat answers of any model we tested.
Qwen3.5 9B	5.7 GB	Did well in chat testing, and digs a little deeper into code before answering.
Gemma 4 E4B	5.0 GB	Did well in our chat testing.
Gemma 4 E2B	3.1 GB	OnBoard's default model. Good at both chat and code summaries.
Qwen3.5 2B	1.3 GB	Writes good code summaries remarkably fast, but struggled in our chat testing.

Files, licensing, and provenance

File	Model	Company (Country)	License	Source
gpt-oss-120b-Q4_K_M-0000?-of-00002.gguf	gpt-oss-120b	OpenAI (United States)	Apache-2.0	unsloth/gpt-oss-120b-GGUF
Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf	Qwen3.6-35B-A3B	Alibaba (China)	Apache-2.0	unsloth/Qwen3.6-35B-A3B-GGUF
gemma-4-12b-it-Q4_K_M.gguf	Gemma 4 12B	Google (United States)	Apache-2.0	unsloth/gemma-4-12b-it-GGUF
Qwen3.5-9B-Q4_K_M.gguf	Qwen3.5 9B	Alibaba (China)	Apache-2.0	unsloth/Qwen3.5-9B-GGUF
gemma-4-E4B-it-Q4_K_M.gguf	Gemma 4 E4B	Google (United States)	Apache-2.0	unsloth/gemma-4-E4B-it-GGUF
gemma-4-E2B-it-Q4_K_M.gguf	Gemma 4 E2B	Google (United States)	Apache-2.0	unsloth/gemma-4-E2B-it-GGUF
Qwen3.5-2B-Q4_K_M.gguf	Qwen3.5 2B	Alibaba (China)	Apache-2.0	unsloth/Qwen3.5-2B-GGUF

Quantization policy: Q4_K_M for models ≤12B; Unsloth Dynamic (UD-Q4_K_XL) for the large MoE, where the dynamic quant's quality edge matters most. gpt-oss-120b is a two-part split: download both parts into the same directory and point the server at part 1 — llama.cpp finds the second part automatically.

All models retain their original Apache-2.0 licenses; no modifications were made beyond mirroring the original GGUF files.

Downloads last month: 81

GGUF

Model size

12B params

Architecture

gemma4

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support