Gen-Z-Qwen3.5-9B

Gen-Z-Qwen3.5-9B is a text model for short, informal English conversations. The model uses Qwen3.5-9B as its base. It usually gives short answers with lowercase letters. It can also answer questions about code, mathematics, and general information.

This repository contains the complete fused model in MLX 4-bit format. The weights include the trained adapter changes. You do not need a separate adapter or base model download.

Model files

Item	Value
Format	MLX Safetensors
Quantization	4-bit affine, group size 64
Weight files	1
Weight file size	5.04 GB
Selected adapter	Identity v16, step 64
Training base	mlx-community/Qwen3.5-9B-MLX-4bit
Export software	MLX-LM 0.31.3
Input and output	Text

Some small parameters keep BF16 or F32 storage. These files require an MLX runtime on Apple silicon.

The bf16 branch contains the original fused BF16 model. That model uses 17.9 GB for its weight files. It contains dequantized weights from the original 4-bit training base. Dequantization does not restore the original base model precision.

Quantization training

Training used distilled weight quantization, or DWQ. DWQ adjusts quantization scales and offsets to match the fused BF16 model. The 4-bit integer values stay fixed. The BF16 reference model stays fixed. The release uses equal parts of two trained parameter sets.

The training data included conversation style, model identity, facts, instructions, mathematics, code, summaries, and reasoning. The selected release used 64 pilot updates and 512 balanced updates. The second parameter set used 128 additional focused updates. The export contains all required weights.

Use with Unsloth Desktop

The installed Unsloth inference backend kept this model's existing 4-bit quantization. It did not apply runtime quantization again.

1. Unload the old model.
2. Remove the old local download if Unsloth continues to use it.
3. Download the current version of Spakie/Gen-Z-Qwen3.5-9B.
4. Start a new chat.
5. Set temperature to 0.2 and top-p to 0.8 for the tested settings.
6. Disable thinking for short conversation replies.

Leave the system prompt empty to use the tested conversation setup. Use your Hugging Face account access if the repository is private.

Use with MLX-LM

Use a Mac with Apple silicon and sufficient available memory. The model needs more memory than the weight file size during use.

1. Install MLX-LM.

python -m pip install "mlx-lm==0.31.3"

2. Run the model.

mlx_lm.generate \
  --model Spakie/Gen-Z-Qwen3.5-9B \
  --prompt "whats a tpu" \
  --chat-template-config '{"enable_thinking": false}' \
  --temp 0.2 \
  --top-p 0.8 \
  --max-tokens 512

Set enable_thinking to true to request reasoning text. The </think> token marks the end of that text.

Recorded tests

The tests compare this release with the preserved BF16 model. Each pair uses the same prompts and generation settings. The scores are averages of the check scores for each case. Scores apply only to these small test sets.

Test	Cases	BF16	Trained 4-bit
MLX-LM, seed 17	78	94.23%	95.41%
Unsloth backend, seed 17	78	94.66%	93.06%
MLX-LM, seed 29, short tests	63	96.83%	97.35%

The separate test set contained 32 conversations and 5,155 answer tokens. These conversations were excluded from quantization training and checkpoint selection. The models chose the same most likely next token for 92.90% of these tokens. The mean KL divergence was 0.03922. KL divergence measures the difference between the output probability distributions. A lower value means a closer match.

The Unsloth tests used the installed inference backend. They did not automate the desktop interface. The installed versions were Unsloth 2026.9.4, Unsloth Zoo 2026.9.3, and MLX-VLM 0.6.17.

See evaluation_summary.json for category scores and the weight hash.

Limitations

Individual answers can differ from BF16 answers. Some thinking-mode identity answers invent version details. The model can give incorrect facts, code, or calculations. The model can omit useful details from short answers. The tests do not establish reliable use of long input text or other languages.

License and references

This model uses the Apache 2.0 license.

• Qwen3.5-9B base model
• 4-bit training base
• MLX-LM quantization training