hermitdave/NeoHorse-1-9B-8bit

NeoHorse-1-9B MLX Quantized Variants

This repository contains four MLX-quantized variants of TokenRhythm/NeoHorse-1-9B, a 9B-parameter language model fine-tuned from Qwen3.5-9B for agentic post-training.

Model Details

Property	Value
Model family	NeoHorse Agent-Native Causal Language Model
Parameters	~9B
Base model	Qwen3.5-9B
Architecture	Qwen3.5 hybrid (gated-delta linear attention + full attention)
Context length	262,144 tokens native, extensible to 1,010,000
Weight format	Safetensors / BF16 (source)
License	Apache 2.0

Quantization Variants

Variant	Size	Method	Description
oQ4e	~5.3 GB	oMLX imatrix-enhanced	Mixed-precision 4-bit, sensitivity-calibrated
oQ6e	~7.5 GB	oMLX imatrix-enhanced	Mixed-precision 6-bit, near-lossless
6-bit	~7.3 GB	mlx_lm.convert	Uniform 6-bit affine
8-bit	~9.5 GB	mlx_lm.convert	Uniform 8-bit affine

Usage

mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("hermitdave/NeoHorse-1-9B-oQ4e")

messages = [{"role": "user", "content": "Write a Python function that returns the first n Fibonacci numbers."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)

oMLX

omlx serve hermitdave/NeoHorse-1-9B-oQ4e --port 8000

CLI

python -m mlx_lm.chat --model hermitdave/NeoHorse-1-9B-oQ4e

Evaluation

NeoHorse-1-9B achieves a 69.04 macro average across ten benchmarks vs 65.60 for Qwen3.5-9B (+3.44). See the original model card for full evaluation details.

Conversion Notes

• oQ4e / oQ6e: Quantized using oMLX quantize_oq_streaming with imatrix calibration (128 samples, 512 seq length). These are mixed-precision formats — bit width varies per layer based on sensitivity measurement.
• 6-bit / 8-bit: Quantized using mlx_lm.convert with uniform affine quantization (group_size=64).
• MTP: The source checkpoint declares mtp_num_hidden_layers: 1 but contains no mtp.* weights (dropped during NeoHorse's repackaging from Qwen3.5-9B). These models do not include MTP heads.

Citation

@misc{neohorse2026,
  title        = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness},
  author       = {NeoHorse Team},
  year         = {2026},
  howpublished = {arXiv preprint}
}

Acknowledgments

• TokenRhythm for the NeoHorse-1-9B model
• Qwen for the Qwen3.5-9B base model
• oMLX for the oQ quantization pipeline
• mlx-lm for uniform quantization
• Hermes Agent for conversion orchestration