nace-ai/drex-max-v1-Q8_0

Drex Max Q8_0 GGUF

A Q8_0 GGUF of Drex Max, a decision model that takes a state and named typed questions and returns a probability for every option. It covers the text backbone; images need the Python weights. Size and hash are in the table below.

Drex Max · Drex repository · llama.cpp fork (branch drex-max-v1) · Ollama fork (branch drex-max-v1)

Run with llama.cpp

Build the drex-max-v1 branch of the llama.cpp fork as described on the model page. Then download the GGUF and start the server:

hf is the Hugging Face CLI (pip install -U huggingface_hub).

hf download nace-ai/drex-max-v1-Q8_0 drex-max-v1-Q8_0.gguf --local-dir weights/drex-max-v1-Q8_0

llama.cpp/build/bin/llama-server -m weights/drex-max-v1-Q8_0/drex-max-v1-Q8_0.gguf \
  -np 1 -c 16384 -b 16384 -ub 16384 --no-warmup --port 8000

The server answers POST /v1/systemone; the request format is in the repository docs. Drex Max reads each request in one batch, so -b and -ub equal -c, and one request holds up to that many tokens. For larger windows see Context length.

Run with Ollama

Use the Nace fork of Ollama from the model page, with this Modelfile next to the GGUF:

FROM ./drex-max-v1-Q8_0.gguf
CAPABILITY decision
PARAMETER num_ctx 16384
PARAMETER num_gpu 40

ollama create drex-max-v1 -f Modelfile
curl localhost:11434/v1/systemone -d '{"model": "drex-max-v1", "state": "...", "questions": {...}}'

Kapitsa

Kapitsa v0.2.25 or later imports this GGUF and serves it. hf is the Hugging Face CLI (pip install -U huggingface_hub).

brew install nace-ai/tap/kapitsa
hf download nace-ai/drex-max-v1-Q8_0 drex-max-v1-Q8_0.gguf --local-dir weights/drex-max-v1-Q8_0
export KAPITSA_HOME="$HOME/.cache/kapitsa"
GGUF=weights/drex-max-v1-Q8_0/drex-max-v1-Q8_0.gguf
kapitsa import "$GGUF" --name drex-max-v1
kapitsa-drex-pointer "$GGUF" "$KAPITSA_HOME/models/drex-max-v1/pointer.bin"
kapitsa-systemone --model "$KAPITSA_HOME/models/drex-max-v1" --port 8000 --context 2048

The Q8_0 GGUF needs 48 GB. On a 48 GB Mac, use --context 2048. The affine Q4 checkpoint runs on Apple silicon Macs with 24 GB of memory or more (Kapitsa and MLX): kapitsa-systemone --model ./drex-max-v1-affine-q4 --port 8000 --context 2048.

File

File	Size	SHA-256
drex-max-v1-Q8_0.gguf	27,250,125,216 bytes	083fe3253f15592554c4ccf2de61575bfa46d19ca4515516c87be1a4fb40db7b

The GGUF has causal attention off. It holds 497 Q8_0 tensors and 355 F32 tensors, including pointer.readout.weight and pointer.temperature.weight.

Tested

On an Apple M5 Pro (48 GB unified memory), llama-server and Ollama (num_gpu 40) on Metal return the same answers on the same request, and match the bf16 weights run with Python on an H100 to within 0.02 on every probability.

License

Nace.AI Open RAIL-M license, see LICENSE.