Vontra/Ling-3.0-flash-MLX-4bit

Ling 3.0 Flash — MLX 4-bit

A native Apple-silicon conversion of inclusionAI/Ling-3.0-flash, quantized with the stock MLX-LM converter and packaged for MLX-LM and oMLX.

Original model · InclusionAI on ModelScope · OpenRouter

About this conversion

This repository contains a 4-bit affine MLX conversion of Ling 3.0 Flash. Ling is a 124B-parameter hybrid-linear mixture-of-experts model with approximately 5.1B active parameters per token. The conversion preserves the upstream tokenizer and chat template and includes an MLX architecture adapter for the model's KDA, MLA, and sparse-MoE layers.

Item	Value
Base model	inclusionAI/Ling-3.0-flash
Format	MLX safetensors
Quantization	4-bit affine, group size 64
Converter	Stock mlx_lm.convert from MLX-LM 0.31.3
Effective weight precision	4.502 bits per weight
Repository size	70.03 GB (65.22 GiB)
Maximum configured context	262,144 tokens
Architecture	bailing_hybrid

This model includes the custom architecture file bailing_hybrid.py. Review the file and enable Trust Remote Code before loading it. In oMLX, enable Trust Remote Code in the model settings. With MLX-LM, pass --trust-remote-code.

Apple-silicon validation

The converted weights were loaded strictly with both stock MLX-LM and the MLX runtime bundled with oMLX. End-to-end generation was also tested through the oMLX OpenAI-compatible API on an Apple-silicon Mac Studio.

One short local smoke test produced:

Measurement	Result
Prefill	21.38 tokens/s
Decode	76.41 tokens/s
Reported peak memory	70.18 GB
oMLX API	HTTP 200, valid completion

These figures confirm that the model runs; they are not a controlled benchmark. Prompt length, sampler settings, macOS memory pressure, thermal state, and MLX/oMLX versions can materially change performance.

Quick start with MLX-LM

Install recent MLX-LM and Hugging Face tooling:

python -m pip install -U mlx-lm huggingface_hub

Run a chat prompt directly from the Hub:

mlx_lm.generate \
  --model Vontra/Ling-3.0-flash-MLX-4bit \
  --trust-remote-code \
  --prompt "Explain why hybrid linear attention is useful." \
  --max-tokens 512 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20

Thinking mode is enabled by the upstream chat template by default. It can be disabled with:

mlx_lm.generate \
  --model Vontra/Ling-3.0-flash-MLX-4bit \
  --trust-remote-code \
  --chat-template-config '{"enable_thinking": false}' \
  --prompt "Write a short hello-world program in Swift."

To download the repository first:

hf download Vontra/Ling-3.0-flash-MLX-4bit \
  --local-dir ~/.omlx/models/Vontra/Ling-3.0-flash-MLX-4bit

Using it with oMLX

1. Place the model at ~/.omlx/models/Vontra/Ling-3.0-flash-MLX-4bit.
2. Refresh the oMLX model registry.
3. Open the model settings and enable Trust Remote Code.
4. Load Ling-3.0-flash-MLX-4bit and use the normal chat or OpenAI-compatible endpoint.

Example request:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMLX_API_KEY" \
  -d '{
    "model": "Ling-3.0-flash-MLX-4bit",
    "messages": [{"role": "user", "content": "Say hello from MLX."}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 128
  }'

For long prompts, begin with a conservative context limit and increase it while watching memory pressure. The advertised 256K context is a model capability, not a promise that every host can prefill that context within its available unified memory.

Architecture

Ling 3.0 Flash is a native hybrid-linear MoE model. It alternates Kimi Delta Attention (KDA) and gated Multi-head Latent Attention (MLA) in a 5:1 ratio and uses highly sparse routed experts.

Architecture detail	Upstream value
Total / active parameters	124B / 5.1B
Transformer layers	35 KDA + 7 gated MLA
Dense layers	2
Routed / shared experts	512 / 1
Active routed experts	8
Attention heads	32
Hidden size	2,560
Expert intermediate size	768
Dense intermediate size	6,144
Vocabulary size	157,184
Context training schedule	8K → 32K → 256K

The included MLX adapter uses MLX-LM's native primitives for delta attention, absorbed MLA projections, RoPE, and quantized SwitchGLU experts. The auxiliary MTP training head is intentionally excluded from ordinary causal generation; this release does not claim MTP/DSpark speculative decoding support.

Upstream model highlights

InclusionAI describes Ling 3.0 Flash as a next-generation hybrid reasoning model designed for efficient long-context inference and agentic workloads. The upstream release reports strong results across software-engineering agents, tool use, deep research, general knowledge, mathematical reasoning, instruction following, and long-context understanding.

The upstream defaults are:

thinking: enabled
temperature: 0.6
top_p: 0.95
top_k: 20

For benchmark methodology, scores, limitations, and framework-specific SGLang/vLLM deployment instructions, see the original InclusionAI model card.

Conversion notes

• Source weights: BF16 upstream checkpoint.
• Conversion command: stock mlx_lm.convert with --quantize --q-bits 4 --q-group-size 64 --q-mode affine.
• The upstream chat_template.jinja is included unchanged.
• All source tensors required for base causal generation were consumed by a strict load before conversion.
• The 42 base transformer layers are retained. The separate depth-5 MTP training/speculation stack is not part of this runtime adapter.

Four-bit weight quantization materially reduces memory use but may reduce output quality relative to BF16 or the 8-bit conversion.

This is a community conversion, not an official InclusionAI release. Because the architecture adapter is new, validate quality and numerical behavior on your own representative workload before production use.

License and attribution

The upstream model is released under the MIT License. This conversion preserves that license and is derived from inclusionAI/Ling-3.0-flash.

All model design, training, and benchmark credit belongs to InclusionAI and the original contributors. The MLX conversion and compatibility adapter are provided by Vontra.