Instructions to use Thox-ai/ThoxMini-Phi4Mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Thox-ai/ThoxMini-Phi4Mini with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Thox-ai/ThoxMini-Phi4Mini", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Thox-ai/ThoxMini-Phi4Mini", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Thox-ai/ThoxMini-Phi4Mini", trust_remote_code=True, device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Thox-ai/ThoxMini-Phi4Mini with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Use Docker

docker model run hf.co/Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use Thox-ai/ThoxMini-Phi4Mini with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Thox-ai/ThoxMini-Phi4Mini"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Thox-ai/ThoxMini-Phi4Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

SGLang

How to use Thox-ai/ThoxMini-Phi4Mini with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Thox-ai/ThoxMini-Phi4Mini" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Thox-ai/ThoxMini-Phi4Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Thox-ai/ThoxMini-Phi4Mini" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Thox-ai/ThoxMini-Phi4Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use Thox-ai/ThoxMini-Phi4Mini with Ollama:
```
ollama run hf.co/Thox-ai/ThoxMini-Phi4Mini:Q4_K_M
```
Unsloth Desktop

How to use Thox-ai/ThoxMini-Phi4Mini with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Thox-ai/ThoxMini-Phi4Mini:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Thox-ai/ThoxMini-Phi4Mini with Docker Model Runner:
```
docker model run hf.co/Thox-ai/ThoxMini-Phi4Mini:Q4_K_M
```

Lemonade

How to use Thox-ai/ThoxMini-Phi4Mini with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Run and chat with the model

lemonade run user.ThoxMini-Phi4Mini-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Thox-ai/ThoxMini-Phi4Mini with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Thox-ai/ThoxMini-Phi4Mini with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Thox-ai/ThoxMini-Phi4Mini:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Thox-ai/ThoxMini-Phi4Mini:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

ThoxMini-Phi4Mini — ThoxMini flagship (Phi-4-mini, merged weights)

Your AI. Your Data. Your Rules.™

Summary

ThoxMini-Phi4Mini is the flagship assistant model for the ThoxMini, and the host flagship payload for ThoxKey / ThoxBeam. This repo carries the merged full-weights build of its LoRA adapter (r32) for microsoft/Phi-4-mini-instruct, fine-tuned on the THOX Wave B factuality corpus (982 rows) so the model answers THOX product and company questions from canonical facts, declines what THOX has not published, and stops instead of rambling into fabrication. The merged-weights source repo (Thox-ai/ThoxMini-Phi4Mini-Merged) is retained as the build record.

Phi-4-mini is Tommy's settled flagship pick for this slot: it won the THOX evaluation and the 2026-10-28 thinking-off retest on task completion and tool routing, and it ships in the deployed configuration — merged adapter plus the grounded device-facts system prompt, never bare.

ThoxKey / ThoxBeam role. ThoxKey runs no assistant model on-device; it is mesh-served, and the host (ThoxBeam) runs the model. This build is that host payload, so one flagship covers both the ThoxMini slot and the ThoxKey slot. The on-key role model (ThoxNano-135M tier) is unchanged and is not replaced by this build.

THOX.ai is live on Kickstarter

Explore ThoxKey, ThoxAir, ThoxMini and ThoxClip, and help bring the THOX personal AI ecosystem to life.

Back THOX.ai on Kickstarter

Your AI. Your Data. Your Rules.™

Files

File	Size	Purpose
`model-00001-of-00002.safetensors`	4,903,637,712 B	Model weights (safetensors)
`ThoxMini-Phi4Mini.Q8_0.gguf`	4,084,611,264 B	GGUF model build
`model-00002-of-00002.safetensors`	2,768,428,504 B	Model weights (safetensors)
`ThoxMini-Phi4Mini.Q4_K_M.gguf`	2,493,840,576 B	GGUF model build
`tokenizer.json`	15,524,095 B	Tokenizer files
`vocab.json`	3,910,310 B	Tokenizer files
`merges.txt`	2,418,348 B	Tokenizer files
`thox_gate49.json`	67,038 B	JSON metadata
`model.safetensors.index.json`	16,273 B	JSON metadata
`Modelfile`	6,500 B	Ollama model definition
`training_args.bin`	5,624 B	Repository file
`thox_flagship_probe.json`	3,086 B	JSON metadata
`tokenizer_config.json`	2,962 B	Tokenizer files
`config.json`	2,555 B	Model configuration
`.gitattributes`	1,700 B	Repository file
`special_tokens_map.json`	587 B	Tokenizer files
`added_tokens.json`	249 B	JSON metadata
`generation_config.json`	169 B	Generation configuration

Run it

ollama run Thox-ai/thoxmini-phi4mini

hf download Thox-ai/ThoxMini-Phi4Mini ThoxMini-Phi4Mini.Q4_K_M.gguf --local-dir ./ThoxMini-Phi4Mini

Intended use

On-device assistant for ThoxMini: THOX product questions, device help, tool routing, and general small-model chat in the THOX house voice (terse, technical, no marketing register).
Host-side assistant payload for ThoxKey via ThoxBeam / MeshStack serving.
Always served with the grounded device-facts system prompt (grounding_system.txt, facts card v1.1.0, in the thox-compute plane).

Out of scope: open-ended long-form generation, medical or legal advice, any claim about THOX facts beyond the published record (the model is trained to decline those), and bare deployment without the grounding prompt.

Training data

Dataset: Thox-ai/thox-factory-sft-v5, file train_wave_b.jsonl — 982 rows, each {"messages": [system, user, assistant], "meta": {...}}.
Corpus design: canonical THOX facts (verified facts, campaign facts in campaign-framed questions only), decline behaviour for unpublished information (unknown and pricing categories), and corrections of known-false claims. Rows asserting unpublished chipset identities were excluded during Wave B construction; generic price questions train a decline plus a pointer to thox.ai.
No user data, no personal data, no third-party text corpora.

Training procedure

Method: QLoRA SFT (4-bit NF4 base, LoRA on all linear layers), full-prompt loss (the pinned deps profile: torch 2.4.0 image, transformers 4.51.3, trl 0.15.2, peft 0.14.0 — trl 0.15.2 does not expose completion-only loss). The pinned profile is mandatory for this base: its remote model code imports LossKwargs, removed from every transformers 5.x.
LoRA: r=32, alpha=64, dropout 0.0.
Optimizer/schedule: learning rate 1e-4, cosine, warmup 10 steps (≈5% of the 186-step run), 3 epochs, effective batch 16 (micro-batch 4 × grad accum 4), sequence length 1024.
Infrastructure: HF Jobs under the Thox-ai namespace, flavor a10g-large, dispatched by the THOX compute plane (tommytracx/thox-compute, registry id mini-phi4-flagship).
Output: this repo holds the merged full weights (bf16 safetensors) plus the GGUF conversions and tokenizer. The merge ran as HF Job 6ae2ba083fb22a235dd4f945 (bf16 merge_and_unload, standalone reload smoke-checked in-job before upload).

Evaluation

Publishing authority is the fleet 49-question factuality gate (questions in verified / unknown / known_false / pricing categories, trait-scored 0-5 per question, hard fail on any known-false repetition), run weights-only in Transformers against base + this adapter — see GATE49.md in the launch pack. This model has no prior baseline; this run establishes it. Promotion additionally requires zero known_false hard failures and a full pass on the 6-prompt flagship probe run in the deployed configuration (grounding prompt attached): two grounding traps (chipset identity, unpublished spec), grounded recall, arithmetic, tool-routing shape, and stop behaviour.

49-question gate: 47/49 passed, mean 4.837/5 (gate job 6ae2a9ba41e90535df781336, 2026-10-28; full report in thox_gate49.json in this repo). By category: verified 11/12, unknown 25/26, known_false 10/10, pricing 1/1. One hard failure: on funding, the answer declines correctly, then quotes a fabricated team-page line ("raised approximately $7 million").
6-prompt flagship probe: 3/6 (probe job 6ae2a9bc3fb22a235dd4e861; thox_flagship_probe.json in this repo). Passed: chipset trap, grounded recall, stop behaviour. Failed: spec trap (asserted a Snapdragon 636 chipset identity and unpublished 3,000 mAh / 4 GB carrier specs), arithmetic (17+25 answered 32, wrapped in a THOX-number decline — decline-skin bleed into a general task), tool routing (prose preamble before the call).
Plane 12-prompt gate: not run (the 49-question gate + probe are the publishing authority).

Status: BASELINE ESTABLISHED — NOT PROMOTED

The gate score ties the fleet's best (ThoxEdge-0.5B Wave B, 47/49), but promotion requires a full probe pass, and the probe is 3/6. The three probe failures plus the funding fabrication are specific, corpus-fixable behaviours: mid-decline fabricated figures, spec-trap leakage under the grounding prompt, and decline behaviour bleeding into general arithmetic and tool-call shape. Fix path: Wave C.1 corpus revision (contrast rows for general tasks, figure-free funding declines, spec declines under varied phrasing), retrain, re-gate. Ruling: Sadie Weaving, CAO, 2026-10-28 (reversible by Tommy).

GGUF artifacts

Converted from these merged weights (llama.cpp convert_hf_to_gguf.py + llama-quantize; the f16 intermediate is retained on the build host):

ThoxMini-Phi4Mini.Q8_0.gguf — 4,084,611,264 B, sha256 b019051fe8e07a9f5c709c4eed021a7e17c997144e901bef530f0267a8ed8f9a
ThoxMini-Phi4Mini.Q4_K_M.gguf — 2,493,840,576 B, sha256 a19e96061d84ca2014d4de98d0c0397e6df4db8f73c2163afa6bcb90d15afedf
Modelfile (this repo) builds the Ollama artifact from the Q4_K_M GGUF (Phi-4 chat template, <|end|> stop, and the full grounding system prompt — the same configuration the flagship probe ran).

Ollama: PUSHED 2026-10-29 (CT): thox-ai/thoxmini-phi4mini:q4_k_m and :latest are live on ollama.com. Verified against the registry after the push: the served manifest's model layer is this build's Q4_K_M GGUF (sha256 a19e9606..., 2,493,840,576 B), and a fresh pull into a clean store reproduces this repo's Modelfile modulo ollama show rendering. These artifacts carry the same status as the weights: BASELINE — NOT PROMOTED.

Limitations

A 3.8B-class model: capability comes from the base; the adapter is a facts + behaviour skin, not a capability upgrade.
Bare (ungrounded) deployment is a known-bad configuration: the base model family invents device specs when trapped. Serve with the grounding prompt.
Gate FAIL results, if any, will be stated verbatim on this card; THOX does not publish accuracy claims beyond the gate and probe.

License

MIT, inherited from the base model (microsoft/Phi-4-mini-instruct, MIT). The adapter is a derivative and cannot be freer than its base. THOX training data and procedure are THOX's; the base weights remain Microsoft's under MIT terms.

Provenance

Base model: microsoft/Phi-4-mini-instruct (MIT)
Dataset: Thox-ai/thox-factory-sft-v5 @ train_wave_b.jsonl (982 rows)
Trainer: tommytracx/thox-compute train_lora.py via HF Jobs
Registry entry: mini-phi4-flagship (jobs-flagship.yaml, Wave C)
HF Job ID: 6ae2a6f13fb22a235dd4e5a1 (training, COMPLETED)
Gate job ID: 6ae2a9ba41e90535df781336; probe job 6ae2a9bc3fb22a235dd4e861
Merge job ID: 6ae2ba083fb22a235dd4f945
Trained: 2026-10-28 (HF Jobs, a10g-large, deps profile pinned)
Card status: created at repo creation per the THOX model card rule; evaluation sections filled from the gate results files after the run.

Downloads last month: 11

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for Thox-ai/ThoxMini-Phi4Mini

Base model

microsoft/Phi-4-mini-instruct

Quantized

(203)

this model