rapid-mlx/NeoHorse-1-9B-MLX-4bit

NeoHorse-1-9B MLX 4-bit

This is a 4-bit MLX conversion of TokenRhythm/NeoHorse-1-9B, an agent- and tool-oriented fine-tune of Qwen3.5-9B. It is intended for local text generation on Apple silicon.

The model is text-only. It does not contain the Qwen3.5 vision tower.

Use with Rapid-MLX

rapid-mlx serve rapid-mlx/NeoHorse-1-9B-MLX-4bit --no-mllm

Until a short alias is released, the full repository name can be passed to rapid-mlx serve, rapid-mlx chat, and compatible OpenAI clients.

Use with mlx-lm

from mlx_lm import generate, load

model, tokenizer = load("rapid-mlx/NeoHorse-1-9B-MLX-4bit")
messages = [{"role": "user", "content": "Plan a three-day trip to Kyoto."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

Conversion

Source revision: TokenRhythm/NeoHorse-1-9B@6cd9248d8070d8a0ad8d20aa19e2fe6848419e93.

The checkpoint was converted on Apple silicon with MLX 0.32.2 and mlx-lm 0.31.3 using affine 4-bit quantization and group size 64:

python -m mlx_lm convert \
  --hf-path TokenRhythm/NeoHorse-1-9B \
  --mlx-path NeoHorse-1-9B-MLX-4bit \
  --quantize --q-bits 4 --q-group-size 64

The upstream text-only configuration name was normalized from qwen3_5_text to the equivalent MLX qwen3_5 loader name. The EOS token ID was also aligned with the tokenizer's <|im_end|> token so the terminator is not emitted as visible text.

Local validation

On one Apple-silicon comparison run using identical 4-bit settings and thinking disabled, this conversion matched Qwen3.5-9B on the repository's ten reasoning and ten executable coding cases, scored 7/10 versus 6/10 on its general set, and selected the expected first tool action in 28/30 cases versus 25/30. The tool probe includes parallel-call emission but is not a complete multi-step agent benchmark.

One 150-word generation smoke measured 115.2 tokens/s and 5.25 GB peak memory. This is a single-machine smoke measurement, not a cross-device performance claim.

See the source model card for training, evaluation, intended-use, and limitations information. Quantization can change output quality; independently validate the model for your workload.

License

Apache-2.0. See the source repository and included metadata for applicable notices.