Instructions to use Thox-ai/ThoxWave-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Thox-ai/ThoxWave-8B with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("dphn/dolphin-2.9-llama3-8b")
model = PeftModel.from_pretrained(base_model, "Thox-ai/ThoxWave-8B")

llama-cpp-python

How to use Thox-ai/ThoxWave-8B with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Thox-ai/ThoxWave-8B",
	filename="thoxwave-8b-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Thox-ai/ThoxWave-8B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Thox-ai/ThoxWave-8B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Thox-ai/ThoxWave-8B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Thox-ai/ThoxWave-8B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Thox-ai/ThoxWave-8B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Thox-ai/ThoxWave-8B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Thox-ai/ThoxWave-8B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Thox-ai/ThoxWave-8B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Thox-ai/ThoxWave-8B:Q4_K_M

Use Docker

docker model run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M

LM Studio
Jan

vLLM

How to use Thox-ai/ThoxWave-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Thox-ai/ThoxWave-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Thox-ai/ThoxWave-8B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M

Ollama
How to use Thox-ai/ThoxWave-8B with Ollama:
```
ollama run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M
```

Unsloth Studio

How to use Thox-ai/ThoxWave-8B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Thox-ai/ThoxWave-8B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Thox-ai/ThoxWave-8B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Thox-ai/ThoxWave-8B to start chatting

Atomic Chat new
Docker Model Runner
How to use Thox-ai/ThoxWave-8B with Docker Model Runner:
```
docker model run hf.co/Thox-ai/ThoxWave-8B:Q4_K_M
```

Lemonade

How to use Thox-ai/ThoxWave-8B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Thox-ai/ThoxWave-8B:Q4_K_M

Run and chat with the model

lemonade run user.ThoxWave-8B-Q4_K_M

List all available models

lemonade list

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

ThoxWave-8B

THOX SFT LoRA adapter for the on-device THOX assistant family. Mid-size chat.

At a glance


Base model	`dphn/dolphin-2.9-llama3-8b (Llama-3-8B derivative)`
Parameters	8B
Adapter	LoRA (r=8, alpha=16, target: attention + MLP projections)
Quantization (GGUF)	Q4_K_M
Context	8192
Training data	THOX-generated instruction data, 1,565 rows (SFT)
Final train loss	1.090
License	Meta Llama 3 Community License (inherited from base)
Target device	ThoxNova / ThoxMini

Intended use

On-device THOX assistant workflows — chat, code, documents, design, and device support — tied to the THOX device family. Runs locally via Ollama or llama.cpp; no cloud account required.

Not for

Medical, legal, or financial advice; direct access by minors without adult supervision; safety-critical control loops; automated content moderation without human review.

Training data

Fine-tuned on THOX-generated instruction data (1,565 rows for this SFT model). Base-model pre-training data is governed by the upstream provider's terms; THOX does not claim provenance over it.

Bias & limitations

Honest disclosure: SFT may echo base-model biases. This model is not RLHF'd for harm reduction beyond a light identity persona. Hallucinations are possible. The THOX identity persona is a light overlay and may not fully suppress base-model self-identification.

Prompt template

<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n

Stop token: <|im_end|>

Identity: pinned via the thox-identity.system.md system prompt (embedded as the Ollama SYSTEM directive) plus a thox-identity.filter.py post-filter that rewrites any residual base-vendor self-identification before display.

Quick start

Ollama

ollama pull thox-ai/thoxwave:8b-q4_k_m
ollama run thox-ai/thoxwave:8b-q4_k_m "Who are you?"

GGUF via HF

hf download Thox-ai/ThoxWave-8B --include "*.gguf"

HF Transformers (adapter)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("dphn/dolphin-2.9-llama3-8b", device_map="auto")
model = PeftModel.from_pretrained(base, "Thox-ai/ThoxWave-8B")
tok = AutoTokenizer.from_pretrained("dphn/dolphin-2.9-llama3-8b")

llama.cpp

llama-cli -m ThoxWave-8B-Q4_K_M.gguf -p "Who are you?" -no-cnv -st

Related artifacts

HF: https://huggingface.co/Thox-ai/ThoxWave-8B
Ollama: https://ollama.com/thox-ai/thoxwave
Factory: https://github.com/ttracx/thoxllm-factory
THOX device family: https://github.com/ttracx/thox-device-dependency-map

Changelog

v1 (2026-07-05): initial SFT release; GGUF Q4_K_M; card + legal footer.

Access and distribution

Ollama (public discovery surface). The merged GGUF is freely pullable from the public Thox.ai Ollama profile (see Quick start). Ollama's registry is open, so the Ollama-hosted copy is public.
HuggingFace (gated source of weights). This repository is gated. Download of the GGUF and LoRA adapter requires manual access approval by Thox.ai LLC. See GATED_ACCESS.md for the access-request questionnaire.

License & Legal

License: This adapter is released under the Meta Llama 3 Community License, inherited from the base model. See Meta Llama 3 Community License for exact terms.

No warranty: Provided AS IS without warranty of any kind. Users assume all risk.

Privacy: Thox.ai LLC does not collect user prompts or completions from local inference. See https://thox.ai/privacy for the full policy.

Trademarks: THOX and Thox.ai are trademarks of Thox.ai LLC. Llama is a trademark of Meta Platforms, Inc. Trademark use here does not imply endorsement by the respective owners.

Entity: Thox.ai LLC is an independent Texas limited liability company, and not a DBA of any other entity.

Officers: Tommy Xaypanya - Chief Technology Officer (CTO). Craig Ross - Chief Executive Officer (CEO).

No affiliation: Thox.ai LLC is independent of, and not affiliated with, endorsed by, or sponsored by the upstream base model owner.

Work product: The THOX-specific LoRA fine-tune, the merged model, and the GGUF quantization and packaging are works of Thox.ai LLC, Copyright (c) 2026 Thox.ai LLC. Thox.ai LLC claims no ownership of the upstream base model weights.

Downloads last month: 52

GGUF

Model size

8B params

Architecture

llama

Hardware compatibility

4-bit

Model tree for Thox-ai/ThoxWave-8B

Base model

meta-llama/Meta-Llama-3-8B

Finetuned

dphn/dolphin-2.9-llama3-8b

Adapter

(5)

this model