yibie/Index-Translate-2B-MLX-4bit

Index-Translate-2B-MLX (4bit)

MLX conversion of IndexTeam/Index-Translate-2B for Apple Silicon — a 2B multilingual translation model (150 languages) built on Qwen3.5, supporting translation instructions about terminology, formatting, style, structure, context and output length.

This repo: 4-bit, group size 64 (4.503 bits/weight) · 1.0 GB on disk · ~1.68 GB peak RAM · ~221 tok/s decode.

Available variants

Variant	Disk	Peak RAM	Decode (long output)	Notes
-bf16	3.5 GB	4.39 GB	~97 tok/s	reference, unquantized
-8bit	1.9 GB	2.63 GB	~155 tok/s	bit-identical output to bf16 in our tests
-6bit	1.4 GB	2.16 GB	~175 tok/s	best size/quality trade-off
-4bit-gs32	1.1 GB	1.81 GB	~205 tok/s	group size 32
-4bit	1.0 GB	1.68 GB	~221 tok/s	group size 64

Quick start

pip install "mlx-lm>=0.31.3"     # qwen3_5 support was added in 0.31.3

The prompt template matters. This model was trained with one fixed translation instruction and greedy decoding. Use it verbatim, and pass enable_thinking=False (otherwise the template emits a think block):

from mlx_lm import load, generate

model, tok = load("yibie/Index-Translate-2B-MLX-8bit")
text, target = "你好，世界。今天天气不错。", "英语"      # 英语 / 中文 / 日语 / 德语 / ...
prompt = tok.apply_chat_template(
    [{"role": "user", "content":
      f"请将以下文本翻译为{target}，直接输出翻译结果，不要进行任何解释。\n\n{text}"}],
    add_generation_prompt=True, tokenize=False, enable_thinking=False)

print(generate(model, tok, prompt=prompt, max_tokens=1024, verbose=False))

OpenAI-compatible server (works with the official client):

mlx_lm.server --model yibie/Index-Translate-2B-MLX-8bit --port 8000

python inference/llm/translate.py "你好，世界" --target en \
  --model yibie/Index-Translate-2B-MLX-8bit --base-url http://127.0.0.1:8000/v1

LM Studio

Works out of the box — LM Studio's MLX runtime 1.11.0 bundles mlx-lm==0.31.3, detects the architecture as qwen3_5, and loads this repo in 8 s / 1.9 GiB. Search the model inside LM Studio, or drop the folder into `/.lmstudio/models//Index-Translate-2B-MLX-8bit/`.

Conversion notes

• Converted with mlx_lm.convert from the original checkpoint (model.safetensors, sha256 2f499211fe35…4c0b60, verified after download).
• Vision tower (≈663 MB) and MTP head (≈122 MB) were dropped. This is a text-translation model, and mlx-lm's qwen3_5 implementation is text-only. All 320/320 text tensors are preserved; lm_head is tied to the embedding matrix (tie_word_embeddings: true), matching the source.
• The source checkpoint stores its RMSNorm weights zero-centred (w - 1). mlx-lm's qwen3_5 sanitizer adds 1.0 back to exactly the affected tensors (input_layernorm, post_attention_layernorm, q_norm, k_norm, final norm) and leaves linear_attn.norm.weight alone (it is a plain ~0.92 scale). We verified the loaded weights land at the expected ~1.0-centred distribution, and that the conversions reproduce the source model's translations.
• The chat template is shipped both as chat_template.jinja (mlx-lm) and inline in tokenizer_config.json (other runtimes such as LM Studio).

• Quantization: affine 4-bit, group size 64 (4.503 bits/weight, mlx_lm.convert -q --q-bits 4 --q-group-size 64)
• Base architecture: model_type: qwen3_5, 24 layers (18 Gated DeltaNet linear-attention + 6 full-attention), hidden 2048, head_dim 256
• Requires mlx-lm >= 0.31.3; LM Studio MLX runtime >= 1.11.0
• Source model license: Apache-2.0