Gemma-4-E4B-dailydialog MLX 6bit
6bit MLX quantization of supersamdev/Gemma-4-E4B-dailydialog-16bit, a Gemma 4 4B VLM fine-tuned on DailyDialog.
Requires Apple Silicon (M1/M2/M3/M4). RAM needed: ~8 GB.
Usage
pip install mlx-vlm
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config
model_path = "supersamdev/Gemma-4-E4B-dailydialog-MLX-6bit"
model, processor = load(model_path)
config = load_config(model_path)
prompt = apply_chat_template(processor, config, "Hello!", num_images=0)
response = generate(model, processor, prompt, max_tokens=200, verbose=True)
Other Quants
| Repo | Bits | RAM |
|---|---|---|
| Gemma-4-E4B-dailydialog-MLX-2bit | 2-bit | ~3 GB |
| Gemma-4-E4B-dailydialog-MLX-4bit | 4-bit | ~6 GB |
| Gemma-4-E4B-dailydialog-MLX-6bit | 6-bit | ~8 GB |
| Gemma-4-E4B-dailydialog-MLX-8bit | 8-bit | ~10 GB |
Parent Model
supersamdev/Gemma-4-E4B-dailydialog-16bit
Quantization
Converted using mlx-vlm on Apple Silicon.
- Downloads last month
- 14
Model size
2B params
Tensor type
BF16
·
U32 ·
Hardware compatibility
Log In to add your hardware
6-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for supersamdev/Gemma-4-E4B-dailydialog-MLX-6bit
Base model
google/gemma-4-E4B Finetuned
google/gemma-4-E4B-it