PocketAiHub/PocketAI-Qwen3.6-27B-MLX

PocketAI Qwen3.6-27B MLX

Official-source MLX releases of Qwen/Qwen3.6-27B, converted and validated by PocketAI Model Lab. This repository contains compact 4-bit, higher-precision 8-bit, and full BF16 variants derived from the same pinned official revision.

Variants

Variant	Folder	Stored size	Precision layout
MLX 4-bit	4bit/	16,081,491,566 bytes (14.98 GiB)	498 language modules affine Q4/group 64; vision tower BF16
MLX 8-bit	8bit/	29,528,168,696 bytes (27.50 GiB)	498 language modules affine Q8/group 64; vision tower BF16
MLX BF16	bf16/	54,740,454,051 bytes (50.98 GiB)	All 1,184 stored tensors BF16

The effective stored precisions reported by the converter are 4.695 bits per weight for the 4-bit release and 8.627 bits per weight for the 8-bit release. The unquantized vision tower accounts for the difference from a purely language-only bits-per-weight estimate.

Download and load

Install the validated runtime on an Apple Silicon Mac:

python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"

Download only the desired variant and load its local subfolder:

from pathlib import Path

from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

repo_id = "PocketAiHub/PocketAI-Qwen3.6-27B-MLX"
variant = "4bit"  # "4bit", "8bit", or "bf16"

snapshot = Path(
    snapshot_download(
        repo_id,
        allow_patterns=[f"{variant}/*"],
    )
)
model, processor = load(str(snapshot / variant))

prompt = apply_chat_template(
    processor,
    model.config,
    "Explain why seasons occur.",
    num_images=0,
    enable_thinking=False,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    enable_thinking=False,
)
print(result.text)

For vision input, pass an image path to mlx_vlm.generate and build the prompt with num_images=1.

Reproducibility and validation

• Official source: Qwen/Qwen3.6-27B
• Pinned source revision: 6a9e13bd6fc8f0983b9b99948120bc37f49c13e9
• Converter: mlx-vlm==0.6.8
• Base dtype: BF16
• Quantization: MLX affine, group size 64
• Deterministic text smoke: exact POCKETAI_OK
• Deterministic image smoke: exact dominant color red
• Full PocketAI Model Lab suite: 94/94 tests
• Every uploaded variant includes an artifact-manifest.json with exact file sizes and SHA-256 hashes.

The checks above establish artifact integrity, strict runtime loading, basic text generation, and basic image understanding. They are not a broad benchmark or a guarantee of correctness for every prompt, context length, or serving configuration.

License and attribution

These conversions retain the original model's Apache 2.0 license. See LICENSE and the official Qwen model card.

• PocketAI Model Lab: https://github.com/PocketAIHub/pocketai-model-lab