Instructions to use mlx-community/gemma-4-12b-coder-fable5-composer2.5-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/gemma-4-12b-coder-fable5-composer2.5-8bit with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("mlx-community/gemma-4-12b-coder-fable5-composer2.5-8bit") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use mlx-community/gemma-4-12b-coder-fable5-composer2.5-8bit with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "mlx-community/gemma-4-12b-coder-fable5-composer2.5-8bit" --prompt "Once upon a time"
omlx feedback is missing chat template.jinja, unable to chat
Omlx feedback indicates a missing chat template.jinja file, which prevents dialogue.
After I manually downloaded the chat template.jinja file from "https://huggingface.co/yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1/tree/main", omlx could use the model normally.
But there will be problems with tool calls.
If the file is lost, it will affect the 8bit, 4bit, and f16 versions.
Oh I see, let me take a look on this. I might have overlooked on the jinja file. Thanks for pointing it out.
I just downloaded the model today and was getting the same error, I'm glad this community exists.
Omlx feedback indicates a missing chat template.jinja file, which prevents dialogue.
After I manually downloaded the chat template.jinja file from "https://huggingface.co/yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1/tree/main", omlx could use the model normally.
But there will be problems with tool calls.
If the file is lost, it will affect the 8bit, 4bit, and f16 versions.
How did you point your model to the chat template file?
edit: figured it out: just curl it into the model's dir in .omlx