mlx-community/Qwen2.5-7B-Instruct-4bit

The Model mlx-community/Qwen2.5-7B-Instruct-4bit was converted to MLX format from Qwen/Qwen2.5-7B-Instruct using mlx-lm version 0.18.1.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Qwen2.5-7B-Instruct-4bit")
response = generate(model, tokenizer, prompt="hello", verbose=True)

Downloads last month: 12,454

Safetensors

Model size

1.19B params

Tensor type

FP16

U32

Inference Examples

Text Generation

Inference API (serverless) does not yet support mlx models for this pipeline type.

Model tree for mlx-community/Qwen2.5-7B-Instruct-4bit

Base model

Qwen/Qwen2.5-7B

Finetuned

(169)

this model

Collection including mlx-community/Qwen2.5-7B-Instruct-4bit

Qwen2.5

Collection

The Qwen 2.5 models are a series of AI models trained on 18 trillion tokens, supporting 29 languages and offering advanced features such as instructio • 33 items • Updated Oct 12 • 6