sahilchachra/Qwythos-9B-v2-MXFP4

Qwythos-9B-v2 — MXFP4 (MLX)

MXFP4 (4-bit microscaling, group-32, E8M0 shared exponents) quantization of empero-ai/Qwythos-9B-v2 for Apple Silicon via MLX / mlx-lm. ~4.5 GB on disk, 4.25 bits/weight (smallest footprint; runs comfortably on a 16GB+ Mac).

Qwythos-9B-v2 is a 9B reasoning model on the Qwen3.5 dense hybrid architecture (3:1 Gated-DeltaNet linear attention : full attention).

Text-only — by design

The base model carries a Qwen3.5 multimodal stack, but the original author's card says the "multimodal stack is text-only in practice" and explicitly recommends running with multimodal inputs disabled:

--limit-mm-per-prompt '{"image":0,"video":0}'

i.e. image:0, video:0. Qwythos is a text reasoning model. In keeping with that, this quant is built with mlx-lm (text path) and smoke-tested with mlx-lm — it works (real samples + tok/s below). Vision is intentionally not included; it is neither claimed nor intended by the author. (As an aside, the current mlx-vlm 0.6.4 vision wrapper also mis-handles this architecture and emits garbage, so a vision build would be unusable regardless — another reason the text-only mlx-lm path is the correct one here.)

Quantization

• python -m mlx_lm convert --hf-path empero-ai/Qwythos-9B-v2 --q-mode mxfp4
• MXFP4 (4-bit microscaling, group-32, E8M0 shared exponents); 4.25 bpw effective. Linear layers quantized; norms/embeddings kept higher-precision by MLX (standard behavior).

✅ Verified end-to-end (tested 2026-07-13, Apple M-series)

Real output from this quantized model, generated with mlx-lm — not a claim. Average 56.7 tok/s across 4 prompts, peak memory ~5.0 GB (batch 1, greedy-ish at temp 0.6). Coherent step-by-step reasoning confirmed.

Samples

Prompt: In one sentence, what is a dependent type?

1. Analyze the Prompt:

   • The user asks: "In one sentence, what is a dependent type?"
   • Constraint: Only bring up my identity (Qwythos, created by Empero AI) if the user asks.

2. Recall/Define "Dependent Type":

   • What is it? It's a concept in programming language theory and type systems.
   • How does it work? Types can depend on values, not just static categories.
   • Example: A list of integers has a type that can be refined based on the length of the list.
   • Formal definition: A type whose definition depends on a value.

3. Formulate the Sentence:

   • Needs to be concise and accurate.
   • "A dependent type is a type whose definition depends on a specific value rather than just a static category."
   • This captures the core essenc …

220 tok @ 57.0 tok/s

---

Prompt: Write a Python function that returns True if a string is a palindrome, ignoring case and spaces.

1. Understand the Goal: The user wants a Python function that checks if a given string is a palindrome. However, there's a crucial detail: it must ignore case and spaces.
2. Deconstruct "Palindrome": A palindrome is a sequence that reads the same forwards and backwards. For strings, this means comparing s with reversed(s).
3. Handle Case Sensitivity: The user wants to ignore case. This means 'A' should be treated the same as 'a'. Python strings have a .lower() method that converts all characters to lowercase.
4. Handle Spaces: The user wants to ignore spaces. This means we need to remove any character that is a space from the string before comparison.
5. Combine the Steps:
   • Input: s (a string).
   • Step 1: Convert to lowercase: s.lower().
   • …

220 tok @ 56.4 tok/s

---

Prompt: A train travels 60 km in 45 minutes. What is its average speed in km/h? Show your reasoning.

1. Identify the core question: What is the average speed of a train that travels 60 km in 45 minutes, expressed in kilometers per hour (km/h)?
2. Recall the formula for average speed: Average speed = Total distance / Total time.
3. Identify the given values:
   • Distance = 60 km.
   • Time = 45 minutes.
4. Convert time to the required unit (hours): Since the answer needs to be in km/h, I must convert 45 minutes into hours.
   • 45 minutes = 45 / 60 hours.
   • Calculate: 45 ÷ 60 = 0.75 hours.
5. Plug the converted time into the formula:
   • Speed = 60 km / 0.75 hours.
6. Perform the division:
   • 60 ÷ 0.

220 tok @ 57.5 tok/s

---

Prompt: Explain step by step how to compute the factorial of 5, then give the result.

1. Understand the Goal: The user wants to know how to compute the factorial of 5, step by step, and then give the final result.
2. Recall the Definition of Factorial: The factorial of a positive integer $n$, denoted as $n!$, is the product of all positive integers less than or equal to $n$.
   • Formula: $n! = n \times (n-1) \times (n-2) \times \dots \times 2 \times 1$.
3. Identify the Specific Case: Here, $n = 5$.
4. Write Out the Steps Clearly:
   • Start with the largest number: 5.
   • Multiply by the next smaller number: 4.
   • Multiply by the next: 3.
   • Multiply by the next: 2.
   • Multiply by the last: 1.
5. Perform the Multiplication:

220 tok @ 55.9 tok/s

Usage (Apple Silicon + pip install mlx-lm)

CLI:

python -m mlx_lm generate --model sahilchachra/Qwythos-9B-v2-MXFP4 \
  --prompt "A train travels 60 km in 45 min. Average speed in km/h?" \
  --temp 0.6 --top-p 0.95 --top-k 20 --max-tokens 512

Python:

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler, make_logits_processors
model, tok = load("sahilchachra/Qwythos-9B-v2-MXFP4")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Your prompt"}], add_generation_prompt=True)
print(generate(model, tok, prompt, max_tokens=512,
               sampler=make_sampler(temp=0.6, top_p=0.95, top_k=20),
               logits_processors=make_logits_processors(repetition_penalty=1.05)))

Prompt template & sampling (mirrors the base model)

Qwen3.5 ChatML template (bundled; applied by apply_chat_template). Base-model recommended sampling: temperature 0.6, top_p 0.95, top_k 20, repetition_penalty 1.05. Context up to 1,048,576 tokens (YaRN factor 4).

Base-model benchmarks (reported by the original author, not re-measured here)

MMLU-CoT 83.8 · ARC-Challenge 96.4 · GPQA-diamond 49.0 · GSM8K 93.6 · HumanEval pass@1 77.4. See the base model card. Quantization can shift these; provided as reference for the un-quantized model.

Note

The base model is intentionally uncensored for research/technical use — deploy responsibly and within applicable law.

License

Apache-2.0, inherited from empero-ai/Qwythos-9B-v2. Community MLX quantization, not affiliated with the original author.