---
license: cc-by-nc-4.0
---

# FlatOrcamaid-13b-v0.2 4bit MLX
MLX quants of [NeverSleep/FlatOrcamaid-13b-v0.2](https://huggingface.co/NeverSleep/FlatOrcamaid-13b-v0.2)

This is an MLX quant of of FlatOrcamaid, MLX is for use with Apple silicon.
The 4bpw fits seems to work well on my 16gig M1 MBP, 8bpw needs more ram.

[Documentation on MLX](https://github.com/ml-explore/mlx/)

### Other Quants:

-MLX: [8bit](https://huggingface.co/Kooten/FlatOrcamaid-13b-v0.2-8bit-mlx), [4bit](https://huggingface.co/Kooten/FlatOrcamaid-13b-v0.2-4bit-mlx)

-Exllama: [8bpw](https://huggingface.co/Kooten/FlatOrcamaid-13b-v0.2-8bpw-exl2), [6bpw](https://huggingface.co/Kooten/FlatOrcamaid-13b-v0.2-6bpw-exl2), [5bpw](https://huggingface.co/Kooten/FlatOrcamaid-13b-v0.2-5bpw-exl2), [4bpw](https://huggingface.co/Kooten/FlatOrcamaid-13b-v0.2-4bpw-exl2)

## Prompt template: Custom format, or Alpaca

### Custom format:
SillyTavern config files: [Context](https://files.catbox.moe/ifmhai.json), [Instruct](https://files.catbox.moe/ttw1l9.json).

### Alpaca:
```
Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{prompt}

### Response:

```

### Contact
Kooten on discord.