Qwen2.5-Coder-3B-Instruct-Arbor-4bit

masato25/Qwen2.5-Coder-3B-Instruct-Arbor-4bit is an Apple MLX 4-bit quantized derivative of Qwen/Qwen2.5-Coder-3B-Instruct.

This conversion is intended for local instruction/chat-style coding assistance on Apple Silicon using MLX / MLX-LM. It preserves the upstream model architecture, tokenizer, and chat template files, while quantizing supported linear weights to 4-bit for a smaller memory footprint.

Attribution and license

• Base model: Qwen/Qwen2.5-Coder-3B-Instruct
• Upstream license: Qwen Research License Agreement (qwen-research)
• Upstream license file: https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct/blob/main/LICENSE
• This repository is a derivative conversion/quantization and is not the original Qwen release.

A NOTICE file is included in this repository. By using, copying, modifying, redistributing, deploying, or making this derivative model available to others, you are responsible for complying with all applicable upstream terms, including the Qwen Research License Agreement and any additional terms, notices, access requirements, usage instructions, export-control, sanctions, or other legal requirements that apply to the upstream model.

If the upstream license, notices, or model-page terms are updated, those upstream terms may impose additional or different obligations. Please review the upstream model page and license before use or redistribution.

No affiliation, sponsorship, endorsement, or trademark grant

This repository is independently prepared and published by the repository owner. It is not affiliated with, sponsored by, approved by, or endorsed by Alibaba Cloud, Qwen, or their affiliates unless they explicitly state otherwise.

The names "Qwen", "Alibaba Cloud", and related marks are used here only for reasonable descriptive attribution and identification of the upstream base model. No trademark license or other rights in those marks are granted by this repository.

Conversion details

• Source: Qwen/Qwen2.5-Coder-3B-Instruct
• Format: MLX / MLX-LM
• Quantization: 4-bit affine weight quantization
• Group size: 64
• Upstream revision: 488639f1ff808d1d3d0ba301aef8c11461451ec5
• Conversion command:

python -m mlx_lm convert \
  --hf-path Qwen/Qwen2.5-Coder-3B-Instruct \
  --mlx-path Qwen2.5-Coder-3B-Instruct-Arbor-4bit \
  -q \
  --q-bits 4 \
  --q-group-size 64

Usage

Install MLX-LM:

pip install -U mlx-lm

Example generation:

python -m mlx_lm.generate \
  --model masato25/Qwen2.5-Coder-3B-Instruct-Arbor-4bit \
  --prompt "Write a Python function that checks whether a string is a palindrome." \
  --max-tokens 256 \
  --temp 0.0

This is the instruction-tuned variant and is generally a better fit than the base Qwen2.5-Coder-3B model for chat-style coding assistance.

Intended use

This quantized derivative is intended for experimentation, research/evaluation, prototyping, coding assistance, and Apple Silicon local inference where permitted by the upstream Qwen Research License Agreement.

You should evaluate whether your intended use is permitted under the upstream license and related terms. This repository does not expand, waive, or modify any upstream restrictions.

Limitations and safety

Quantization can change output quality, numerical behavior, robustness, and safety characteristics compared with the original model. This repository does not claim improved accuracy, safety, bias mitigation, alignment, or suitability for any particular purpose over the upstream Qwen model.

Model outputs may be inaccurate, unsafe, biased, offensive, incomplete, vulnerable, or otherwise unsuitable for your use case. Do not rely on the model as the sole source of truth for code correctness, security, medical, legal, financial, safety-critical, or other high-stakes decisions. Review and test generated code before use.

Disclaimer

This derivative model is provided as-is and without warranties or conditions of any kind, express or implied, including without limitation warranties of merchantability, fitness for a particular purpose, title, non-infringement, accuracy, availability, or error-free operation.

To the maximum extent permitted by applicable law, the repository owner is not liable for any direct, indirect, incidental, special, consequential, exemplary, punitive, or other damages arising from or related to use of this repository, the derivative model, generated code, or other model outputs.

Nothing in this README is legal advice. You are responsible for reviewing and complying with the applicable license terms and laws.