PocketAiHub/Qwen3.8-27B-MLX

Qwen3.8-27B MLX

Community MLX conversions of Qwen/Qwen3.8-27B, produced and validated by PocketAI Model Lab. PocketAI did not train or fine-tune this model. This is not an official Qwen release, and no endorsement is implied.

This repository contains compact 4-bit, balanced 6-bit, higher-precision 8-bit, and unquantized BF16 variants derived from the same pinned official revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.

Variants

Variant	Folder	Stored size	Precision layout
MLX 4-bit	4bit/	16,081,493,231 bytes (14.98 GiB)	498 language modules affine Q4/group 64; vision tower BF16
MLX 6-bit	6bit/	22,804,831,851 bytes (21.24 GiB)	498 language modules affine Q6/group 64; vision tower BF16
MLX 8-bit	8bit/	29,528,170,361 bytes (27.50 GiB)	498 language modules affine Q8/group 64; vision tower BF16
MLX BF16	bf16/	54,740,455,716 bytes (50.98 GiB)	Unquantized BF16 reference

Validation

Every variant passed the same deterministic PocketAI feature suite:

• 12/12 quality regression cases
• 8/8 tool-calling cases
• temporal video understanding (red->blue)
• 4K-context needle retrieval (COBALT-7319)

The 4-bit variant additionally passed a 32K-context retrieval test at 32,770 actual prompt tokens.

4K MLX performance

These are warmed, single-run measurements on an Apple M5 Max with 128 GB unified memory using mlx==0.32.0, mlx-vlm==0.6.8, batch size 1, temperature 0, seed 0, and thinking disabled. The prompt contained 4,105 tokens and the exact retrieval response contained 9 generated tokens.

Variant	Prompt prefill	Generation	End-to-end	Peak MLX memory
MLX 4-bit	733.1 tok/s	36.6 tok/s	5.86 s	21.80 GB
MLX 6-bit	584.7 tok/s	26.2 tok/s	7.38 s	29.54 GB
MLX 8-bit	600.1 tok/s	20.1 tok/s	7.30 s	37.27 GB
MLX BF16	623.8 tok/s	10.8 tok/s	7.43 s	58.29 GB

The full machine-readable evidence is in benchmarks/4k-feature-suite.json. These compact deterministic checks are regression gates, not a broad claim of quality parity across precisions.

Download and load

Install the validated runtime on an Apple Silicon Mac:

python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"

Download only the desired variant and load its local subfolder:

from pathlib import Path

from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

repo_id = "PocketAiHub/Qwen3.8-27B-MLX"
variant = "4bit"  # "4bit", "6bit", "8bit", or "bf16"

snapshot = Path(snapshot_download(repo_id, allow_patterns=[f"{variant}/*"]))
model, processor = load(str(snapshot / variant))

prompt = apply_chat_template(
    processor,
    model.config,
    "Explain why seasons occur.",
    num_images=0,
    enable_thinking=False,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    enable_thinking=False,
)
print(result.text)

For image input, pass an image path to mlx_vlm.generate and build the prompt with num_images=1. Video input was validated with MLX-VLM's video path and FPS arguments.

Reproducibility

• Official source: Qwen/Qwen3.8-27B
• Pinned source revision: 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
• Converter: mlx-vlm==0.6.8
• Base dtype: BF16
• Quantization: MLX affine, group size 64
• Every variant includes an artifact-manifest.json with exact file sizes and SHA-256 hashes.
• release-manifest.json records the release inputs and validation evidence.

License and attribution

These conversions retain the original model's Apache 2.0 license. See LICENSE and the official Qwen model card.

• PocketAI Model Lab: https://github.com/PocketAIHub/pocketai-model-lab