Instructions to use stratorys/zeta-2.1-mlx-mixed-3-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use stratorys/zeta-2.1-mlx-mixed-3-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm
# if on a CUDA device, also pip install mlx[cuda]

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("stratorys/zeta-2.1-mlx-mixed-3-4bit")

prompt = "Once upon a time in"
text = generate(model, tokenizer, prompt=prompt, verbose=True)

Transformers

How to use stratorys/zeta-2.1-mlx-mixed-3-4bit with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="stratorys/zeta-2.1-mlx-mixed-3-4bit")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("stratorys/zeta-2.1-mlx-mixed-3-4bit")
model = AutoModelForCausalLM.from_pretrained("stratorys/zeta-2.1-mlx-mixed-3-4bit", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

vLLM

How to use stratorys/zeta-2.1-mlx-mixed-3-4bit with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "stratorys/zeta-2.1-mlx-mixed-3-4bit"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stratorys/zeta-2.1-mlx-mixed-3-4bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/stratorys/zeta-2.1-mlx-mixed-3-4bit

SGLang

How to use stratorys/zeta-2.1-mlx-mixed-3-4bit with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "stratorys/zeta-2.1-mlx-mixed-3-4bit" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stratorys/zeta-2.1-mlx-mixed-3-4bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "stratorys/zeta-2.1-mlx-mixed-3-4bit" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "stratorys/zeta-2.1-mlx-mixed-3-4bit",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

MLX LM

How to use stratorys/zeta-2.1-mlx-mixed-3-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Generate some text
mlx_lm.generate --model "stratorys/zeta-2.1-mlx-mixed-3-4bit" --prompt "Once upon a time"

Docker Model Runner
How to use stratorys/zeta-2.1-mlx-mixed-3-4bit with Docker Model Runner:
```
docker model run hf.co/stratorys/zeta-2.1-mlx-mixed-3-4bit
```
Atomic Chat

Zeta 2.1 MLX — Mixed 3/4-bit

Mixed 3-bit / 4-bit MLX quantization of zed-industries/zeta-2.1, optimized for local edit prediction on Apple Silicon.

Format: MLX
Quantization: mixed 3-bit / 4-bit
Use case: code edit prediction / autocomplete
Tested on: Apple M4 Pro, 24 GB unified memory
Client: Zed
Server: MLX-LM

This is an independently produced quantized conversion and is not an official Zed Industries release.

Usage

Install MLX-LM:

uv tool install mlx-lm

Run the model:

HF_HUB_DISABLE_TELEMETRY=1 \
mlx_lm.server \
  --model stratorys/zeta-2.1-mlx-mixed-3-4bit \
  --host 127.0.0.1 \
  --port 9090 \
  --prefill-step-size 1024

Zed

Add to settings.json:

{
  "show_edit_predictions": true,
  "edit_predictions": {
    "provider": "open_ai_compatible_api",
    "mode": "eager",
    "open_ai_compatible_api": {
      "api_url": "http://127.0.0.1:9090/v1/completions",
      "model": "stratorys/zeta-2.1-mlx-mixed-3-4bit",
      "prompt_format": "zeta2_1",
      "max_output_tokens": 128,
      "prediction_debounce": 200
    }
  }
}

Offline usage

Once the model is cached:

HF_HUB_OFFLINE=1 \
HF_HUB_DISABLE_TELEMETRY=1 \
mlx_lm.server \
  --model stratorys/zeta-2.1-mlx-mixed-3-4bit \
  --host 127.0.0.1 \
  --port 9090 \
  --prefill-step-size 1024

Quantization

This repository contains a mixed 3-bit / 4-bit MLX quantization of Zeta 2.1.

The goal is to reduce model size, memory bandwidth, and inference cost while preserving edit-prediction quality as much as possible.

No additional fine-tuning was performed.

Limitations

Quantization may slightly reduce prediction quality compared with the original model.

Generated edits may be incorrect or fail to compile and should be reviewed before use.

Zeta 2.1 is specialized for edit prediction and is not intended as a general-purpose chat or agent model.

Attribution

Original model:

zed-industries/zeta-2.1

Upstream base model:

ByteDance-Seed/Seed-Coder-8B-Base

License

The upstream Zeta 2.1 model is distributed under the Apache License 2.0.

This repository is a quantized derivative. Please refer to the upstream model repository for the authoritative license and attribution requirements.

Downloads last month: 50

Safetensors

Model size

8B params

Tensor type

U32

BF16

MLX

Hardware compatibility

4-bit

Model tree for stratorys/zeta-2.1-mlx-mixed-3-4bit

Base model

ByteDance-Seed/Seed-Coder-8B-Base

Finetuned

zed-industries/zeta-2.1

Quantized

(26)

this model