Instructions to use thoddnn/d3-mini-q4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use thoddnn/d3-mini-q4 with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("thoddnn/d3-mini-q4")
config = load_config("thoddnn/d3-mini-q4")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use thoddnn/d3-mini-q4 with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "thoddnn/d3-mini-q4"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "thoddnn/d3-mini-q4"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use thoddnn/d3-mini-q4 with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "thoddnn/d3-mini-q4"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default thoddnn/d3-mini-q4

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use thoddnn/d3-mini-q4 with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "thoddnn/d3-mini-q4"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "thoddnn/d3-mini-q4" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

D3 Mini Q4 for Fenn / MLX

An MLX 4-bit conversion of vllm-sr/d3-mini for local decision verification on Apple silicon.

This is a decision model, not a generative chat model. It requires the Fenn D3 MLX runtime: stock mlx_lm.generate, mlx_vlm.generate, or Transformers generation do not implement its readout and attention contract.

Conversion

Source revision: a475367de0d8133a3c1a2a0dc12303d89cfa2ab4.
Language projections and embeddings: affine 4-bit, group size 64.
Vision tower: bfloat16.
Offset RMS normalization weights and decision readout: float32.
Full-attention layers are noncausal; DeltaNet recurrence remains causal.
Last-token pooling, source decision temperature, and code ordering are retained.
weights.safetensors includes the decision readout; no generative language-model head is included.
fenn_d3.json records the upstream provenance and SHA-256 checksums of runtime assets.

The weights and tokenizer/processor files are self-contained; the original full-precision checkpoint is not needed at runtime. Text-only and single-image-plus-text evidence are supported by Fenn's verifier.

Fenn decisions

Fenn supplies three options in this order: A = MATCH, B = NOT_MATCH, C = UNCERTAIN. Their probabilities are scores conditional on the supplied prompt and options, not calibrated accuracy guarantees. Use the matching runtime and evaluate on your own data before choosing thresholds.

License and attribution

The original model is by the vLLM Semantic Router Team, based on Qwen3.5. The original Apache-2.0 LICENSE and NOTICE are preserved. This repository changes the weight representation and packages the assets for Fenn's MLX runtime; it does not claim additional training or improved accuracy.

Downloads last month: 56

MLX

Hardware compatibility

Quantized

Model tree for thoddnn/d3-mini-q4

Base model

Qwen/Qwen3.5-4B-Base

Finetuned

Qwen/Qwen3.5-4B

Finetuned

vllm-sr/d3-mini

Finetuned

(6)

this model