Falcon-H1 1.5B MLX 4-bit
4-bit quantized MLX port of tiiuae/Falcon-H1-1.5B-Base for Apple Silicon.
For BF16: sdkv2/falcon-h1-1.5b-mlx-bf16
Usage
from mlx_lm import load, generate
model, tokenizer = load("sdkv2/falcon-h1-1.5b-mlx-4bit")
response = generate(model, tokenizer, prompt="Hello", verbose=True)
- Downloads last month
- 46
Model size
0.2B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
4-bit