Instructions to use Roxas13/gemma3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use Roxas13/gemma3 with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("Roxas13/gemma3") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use Roxas13/gemma3 with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "Roxas13/gemma3"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "Roxas13/gemma3" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Roxas13/gemma3", "messages": [ {"role": "user", "content": "Hello"} ] }'
AEGIS Gemma 3 4B IT (text-only, 4-bit MLX)
Text-only, 4-bit MLX-quantized build of Gemma 3 4B IT, used on-device by the AEGIS app. This is a modified (vision tower removed, 4-bit quantized) derivative of Google's Gemma 3 4B IT.
License & notices
Gemma is provided under and subject to the Gemma Terms of Use: https://ai.google.dev/gemma/terms
Use of this model must comply with the Gemma Prohibited Use Policy: https://ai.google.dev/gemma/prohibited_use_policy
This repository redistributes a modified Gemma derivative. By Google's Gemma Terms, the above Terms of Use and Prohibited Use Policy apply to this model and are passed along to all recipients. Modifications relative to the base model: vision encoder removed (text-only) and weights quantized to 4-bit for MLX.
- Downloads last month
- 26
4-bit