Instructions to use AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP")
config = load_config("AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP

Run Hermes

hermes

OpenClaw new

How to use AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

AX Gemma 4 12B IT MLX 6-bit Assistant MTP

Parameter count: approximately 11.96B logical target parameters (12B class). 6-bit is the target quantization precision, not a 6B model-size claim. The separately packaged assistant is an acceleration component and is not included in the target count.

This is a self-contained MLX model package for Apple Silicon. It combines the Gemma 4 12B instruction-tuned target with its exact paired assistant model for AX Engine multi-token prediction (MTP) / speculative decoding.

The target model verifies every drafted token. The assistant improves decode speed without replacing the target model or changing its output distribution. This repository does not contain PyTorch or GGUF target weights.

Model details

Base model: google/gemma-4-12B-it
MLX target: mlx-community/gemma-4-12B-it-6bit, revision f0d6f5d34239a612f695362750044905e6dd072c
Assistant: mlx-community/gemma-4-12B-it-assistant-6bit, revision 69fe060358f645f9231ac9e67451c81794e56b42
Format: MLX Safetensors target plus an AX Engine assistant subtree
Target quantization: 6-bit affine, group size 64; FFN projections retain 8-bit overrides
Assistant quantization: 6-bit affine, group size 64
Configured context length: 131,072 tokens
MTP pairing: exact
Maximum packaged draft depth: 2
Intended hardware: Apple Silicon

The target weight shards and assistant weights are byte-identical to the pinned upstream revisions. AX Engine adds native manifests, an assistant pairing contract, an AX-compatible assistant configuration, and a target-tokenizer copy inside assistant/.

Download

hf download AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP \
  --local-dir ./AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP

The repository is approximately 12.3 GB. Ensure that the destination has enough free space before downloading.

Use with AX Engine MTP

Install AX Engine, then serve the downloaded repository directory:

ax-engine serve ./AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP --port 31418

Gemma assistant MTP is enabled by default. AX Engine reads ax_gemma4_assistant_mtp.json, loads the compact drafter from assistant/, and uses the 12B target to verify proposals. Do not load assistant/ by itself as a general-purpose chat model.

The target weights can also be used for direct MLX generation, but the nested assistant acceleration and pairing contract are AX Engine-specific.

Package layout

model-*.safetensors: 6-bit MLX target weights
model.safetensors.index.json: target shard index
assistant/model.safetensors: paired 6-bit assistant weights
model-manifest.json and assistant/model-manifest.json: AX native manifests
ax_gemma4_assistant_mtp.json: exact-pairing and draft-depth contract
ax_provenance.json: pinned sources and checksums

Validation and provenance

Validated on macOS arm64 with AX Engine 6.9.0 on 2026-07-20:

AX native artifact validation: ready, with no issues
All target weight shards: byte-exact against the pinned MLX source
Assistant weight: byte-exact against the pinned assistant source
Assistant and target tokenizer: byte-identical inside the package
Pairing contract: exact
Canonical chat template: pinned from Google Gemma 4 and applied to the target

See ax_provenance.json for immutable source revisions and SHA-256 values.

License

Apache License 2.0. Review the Gemma terms and the original Google model cards for usage limitations and responsible-use guidance.

Downloads last month: 145

Safetensors

Model size

12B params

Tensor type

BF16

U32

MLX

Hardware compatibility

6-bit

Model tree for AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP

Base model

google/gemma-4-12B

Finetuned

google/gemma-4-12B-it

Quantized

(282)

this model

Collection including AutomatosX/AX-Gemma-4-12B-IT-MLX-6bit-Assistant-MTP

AutomatosX MLX Model Catalog

Collection

Official AutomatosX MLX releases for Apple Silicon. OptiQ mixed 4/8-bit packages sit next to uniform 4bit/6bit siblings. • 31 items • Updated 4 days ago