README.md

---
language:
- en
license: apache-2.0
tags:
- mlx
datasets:
- Locutusque/hercules-v5.0
inference:
  parameters:
    do_sample: true
    temperature: 0.8
    top_p: 0.95
    top_k: 40
    min_p: 0.1
    max_new_tokens: 250
    repetition_penalty: 1.1
---

# mlx-community/Hercules-5.0-Qwen2-1.5B-4bits

The Model [mlx-community/Hercules-5.0-Qwen2-1.5B-4bits](https://huggingface.co/mlx-community/Hercules-5.0-Qwen2-1.5B-4bits) was converted to MLX format from [M4-ai/Hercules-5.0-Qwen2-1.5B](https://huggingface.co/M4-ai/Hercules-5.0-Qwen2-1.5B) using mlx-lm version **0.14.0**.

## Use with mlx

```bash
pip install mlx-lm
```

```python
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Hercules-5.0-Qwen2-1.5B-4bits")
response = generate(model, tokenizer, prompt="hello", verbose=True)
```