Instructions to use Noctalin/Ornith-1.0-9B-oQ8-fp16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Noctalin/Ornith-1.0-9B-oQ8-fp16 with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("Noctalin/Ornith-1.0-9B-oQ8-fp16")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use Noctalin/Ornith-1.0-9B-oQ8-fp16 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-9B-oQ8-fp16"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Noctalin/Ornith-1.0-9B-oQ8-fp16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use Noctalin/Ornith-1.0-9B-oQ8-fp16 with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "Noctalin/Ornith-1.0-9B-oQ8-fp16"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "Noctalin/Ornith-1.0-9B-oQ8-fp16"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "Noctalin/Ornith-1.0-9B-oQ8-fp16",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use Noctalin/Ornith-1.0-9B-oQ8-fp16 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-9B-oQ8-fp16"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Noctalin/Ornith-1.0-9B-oQ8-fp16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Noctalin/Ornith-1.0-9B-oQ8-fp16 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "Noctalin/Ornith-1.0-9B-oQ8-fp16"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Noctalin/Ornith-1.0-9B-oQ8-fp16" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Ornith-1.0-9B oQ8 Text-Only (Optimized for Apple Silicon)

This repository contains a custom-quantized, text-only configuration of the Ornith-1.0-9B dense model, optimized explicitly for local repository-level agentic coding on Apple Silicon using the oMLX inference engine.

🎯 Why This Was Created

Ornith-1.0-9B is a state-of-the-art, self-improving dense model specialized for agentic coding. It jointly optimizes search scaffolds and solution rollouts via Reinforcement Learning to discover superior code trajectories.

This specific build was converted using oMLX v0.4.5.dev1 to address long-context deployment constraints on Apple Silicon:

The Dense High-Fidelity Precision: As a dense 9B model, Ornith-1.0-9B provides outstanding agentic coding performance in a very compact footprint. Quantizing it to oQ8 (8-bit) provides near-lossless code syntax retention and complex instruction adherence, matching the original model's quality while still saving valuable Unified Memory to support deep context windows during complex multi-file repository tasks.
Overcoming the 128k Context Wall: Traditional backends often choke or suffer severe latency degradation when context sizes scale out. Moving to oMLX's specialized two-tier caching eliminates this overhead, allowing you to fluidly ingest giant codebases.
Prefill Speedup via float16: While the base weights are distributed in bfloat16, this build explicitly targets Apple Silicon hardware by using float16 for non-quantized weights, unlocking a ~20% faster prefill speed on M1/M2 Max chips.
MTP Note: Unlike some Qwen base models, the original Ornith-1.0 weights do not contain Multi-Token Prediction (mtp.*) headers. As a result, native MTP decoding is not available for this model.

🚀 Key Differences

Feature / Attribute	Standard Ornith-1.0-9B	This Custom Build (`oQ8-fp16`)
Native MTP Heads	Not present in base architecture	Not Available (No base MTP tensors)
Vision Model (VLM)	N/A (Text-only coding agent)	Stripped/Text-Only
Quantization Method	Standard Uniform / HF / Unsloth	oQ8 (Dynamic mixed-precision calibration)
Non-Quant Weight DType	`bfloat16`	`float16` (~20% faster prefill on M1/M2 Silicon)

💻 Hardware & RAM Recommendations

Mac Hardware Configuration	RAM Recommendation	Status / Performance Expectation
Apple Silicon (Base 16GB / 24GB)	16GB / 24GB Unified Memory	Supported — Fits nicely, though 16GB systems may see context limits. 24GB ensures smooth operations.
M1 / M2 / M3 / M4 (32GB / 36GB / 48GB)	36GB / 48GB Unified Memory	Recommended — Highly stable performance with generous context capacity.
M1 / M2 / M3 / M4 Max / Ultra (64GB+)	64GB+ Unified Memory	Optimal / Best Experience — Optimal setup to fully leverage the near-lossless 8-bit precision alongside large context windows.

🛠️ Quantization Settings

This model was quantized using oMLX v0.4.5.dev1 with the following specification:

Source Model: deepreinforce-ai/Ornith-1.0-9B
Sensitivity Model: None
oQ Level: oQ8
Text Only: Enabled
Preserve MTP weights: Disabled (Not present in source architecture)
Non-quant weight dtype: float16

⚙️ Optimized oMLX Settings (v0.4.5)

To seamlessly route this model through agentic development workspaces like OpenCode, apply the following server specifications in your oMLX dashboard:

Model Basic Settings

Reasoning Parser: qwen3 (Isolates the <think> ... </think> blocks securely away from IDE syntax parsers)
Tool Call Parser: qwen3_xml
CTX Window: 262,144
Max Tokens: 32,768
Temperature: 0.6 (Use 1.0 if attempting to perfectly replicate official benchmark environments)
Top P / Top K: 0.95 / 20
Min P: 0
Repetition / Presence Penalty: 1 / 0

Model Advanced Settings

Enabled Thinking: Checked (True)
Chat Template Kwargs: enable_thinking: true, preserve_thinking: true
Native MTP: Unchecked (False)

Resource Management & Cache

Memory Guard: Aggressive (Forces strict macOS memory/swap cleanup cycles)
Hot Cache Limit (RAM): 40GB (Allocated for hyper-speed Unified Memory history)
Cold Cache Limit (SSD): 371GB (Serialized safetensors storage for context overflow handles)
Max Concurrent Requests: 2 (Protects the 400 GB/s bandwidth bus from degradation)
Embedding Batch Size: 32
Chunked Prefill: Enabled (Prevents instantaneous out-of-memory crashes on massive project context ingestion)
Burst Decode: Aggressive (Coalesces tokens for maximized typing speeds)
Initial Cache Blocks: 256
SSE Keepalive Mode: Chunk

🌡️ Thermal Optimization Notice

Sustained execution across massive context windows heavily taxes the Apple Silicon GPU/CPU complexes, causing rapid heat buildup. Because Apple's default fan profiles emphasize near-silent operation, they delay ramping up system fans until thermal throttling has already begun to affect generation tokens-per-second (TPS).

To protect performance integrity during prolonged coding sessions, it is highly recommended to run a custom fan utility to enforce proactive, aggressive cooling curves:

Source Repository: smc-command on Codeberg

Downloads last month: 64

Safetensors

Model size

9B params

Tensor type

U32

F16

BF16

MLX

Hardware compatibility

8-bit

Model tree for Noctalin/Ornith-1.0-9B-oQ8-fp16

Base model

ornith-ai/Ornith-1.0-9B

Quantized

(104)

this model